> ## Documentation Index
> Fetch the complete documentation index at: https://mintlify.com/primeintellect-ai/verifiers/llms.txt
> Use this file to discover all available pages before exploring further.

# Data Utilities

> Helper functions for dataset processing

# Data Utilities

Utility functions for loading and formatting common benchmark datasets.

<Note>
  These utilities are designed for example datasets and quick prototyping, not core functionality. For production use, load datasets directly using HuggingFace `datasets` library.
</Note>

## Overview

The `verifiers.utils.data_utils` module provides:

* Pre-configured loaders for common benchmarks (GSM8K, MATH, GPQA, etc.)
* Dataset formatting helpers
* Answer extraction utilities
* System prompts for math tasks

## Dataset Loaders

### load\_example\_dataset

```python theme={null}
def load_example_dataset(
    name: str = "gsm8k",
    split: str | None = None,
    n: int | None = None,
    seed: int = 0
) -> Dataset
```

Load a preprocessed benchmark dataset.

<ParamField path="name" type="str" default="gsm8k">
  Dataset name. Supported: `"aime2024"`, `"aime2025"`, `"amc2023"`, `"gpqa_diamond"`, `"gpqa_main"`, `"gsm8k"`, `"math"`, `"math500"`, `"mmlu"`, `"mmlu_pro"`, `"openbookqa"`, `"openrs"`, `"openrs_easy"`, `"openrs_hard"`, `"prime_code"`.
</ParamField>

<ParamField path="split" type="str | None" default="None">
  Dataset split. If None, uses the default split for that dataset (usually "test" or "train").
</ParamField>

<ParamField path="n" type="int | None" default="None">
  Number of examples to load. If None, loads all examples.
</ParamField>

<ParamField path="seed" type="int" default="0">
  Random seed for shuffling when `n` is specified.
</ParamField>

**Returns**: HuggingFace `Dataset` with `question` and `answer` columns.

**Example**:

```python theme={null}
from verifiers.utils.data_utils import load_example_dataset

# Load 100 GSM8K examples
dataset = load_example_dataset("gsm8k", n=100)

# Load all MATH problems
math_dataset = load_example_dataset("math", split="train")

# Load GPQA diamond
gpqa = load_example_dataset("gpqa_diamond")
```

## Formatting Functions

### format\_dataset

```python theme={null}
def format_dataset(
    dataset: Dataset,
    system_prompt: str | None = None,
    few_shot: Messages | None = None,
    question_key: str = "question",
    answer_key: str = "answer",
    map_kwargs: dict = {},
) -> Dataset
```

Add `example_id` and `prompt` columns to a dataset.

<ParamField path="dataset" type="Dataset" required>
  Input dataset to format.
</ParamField>

<ParamField path="system_prompt" type="str | None">
  System prompt to prepend to all prompts.
</ParamField>

<ParamField path="few_shot" type="Messages | None">
  Few-shot examples to include before each question.
</ParamField>

<ParamField path="question_key" type="str" default="question">
  Column name containing questions.
</ParamField>

<ParamField path="answer_key" type="str" default="answer">
  Column name containing answers.
</ParamField>

<ParamField path="map_kwargs" type="dict">
  Additional arguments passed to `dataset.map()`.
</ParamField>

**Returns**: Dataset with `example_id` and `prompt` columns.

**Example**:

```python theme={null}
from verifiers.utils.data_utils import format_dataset, BOXED_SYSTEM_PROMPT
from datasets import load_dataset

# Load raw dataset
raw_dataset = load_dataset("gsm8k", "main", split="test")

# Format with system prompt
formatted = format_dataset(
    raw_dataset,
    system_prompt=BOXED_SYSTEM_PROMPT,
    question_key="question",
    answer_key="answer",
)

# Now has 'prompt' column with messages
print(formatted[0]["prompt"])
# [
#   {"role": "system", "content": "Please reason step by step..."},
#   {"role": "user", "content": "What is 2+2?"}
# ]
```

## Answer Extraction

### extract\_boxed\_answer

```python theme={null}
def extract_boxed_answer(text: str) -> str
```

Extract content from LaTeX `\boxed{}` commands. Finds the last occurrence of `\boxed{...}` in the text and returns the content between matching braces. If no boxed answer is found or braces don't match, returns the original text.

<ParamField path="text" type="str" required>
  Text containing LaTeX boxed answer (e.g., `"\boxed{42}"`).
</ParamField>

**Returns**: `str` - Extracted answer content, or original text if no valid boxed answer found.

**Example**:

```python theme={null}
from verifiers.utils.data_utils import extract_boxed_answer

# Simple answer
text = "The answer is \\boxed{42}"
result = extract_boxed_answer(text)  # "42"

# Nested braces
text = "\\boxed{x = \\frac{1}{2}}"
result = extract_boxed_answer(text)  # "x = \\frac{1}{2}"

# Multiple boxed answers - extracts last one
text = "First \\boxed{A}, then \\boxed{B}"
result = extract_boxed_answer(text)  # "B"

# No boxed answer
text = "Just plain text"
result = extract_boxed_answer(text)  # "Just plain text"
```

### extract\_hash\_answer

```python theme={null}
def extract_hash_answer(text: str) -> str
```

Extract answer after `####` delimiter (GSM8K format). Returns the text after the first `####` marker, stripped of leading/trailing whitespace. If no delimiter is found, returns the original text.

<ParamField path="text" type="str" required>
  Text containing hash-delimited answer (e.g., `"Solution here\n#### 42"`).
</ParamField>

**Returns**: `str` - Answer after `####` delimiter, or original text if no delimiter found.

**Example**:

```python theme={null}
from verifiers.utils.data_utils import extract_hash_answer

# Standard GSM8K format
text = "Step 1: Add them.\nStep 2: Get result.\n#### 42"
result = extract_hash_answer(text)  # "42"

# With spaces
text = "Solution goes here #### 100"
result = extract_hash_answer(text)  # "100"

# No delimiter
text = "Just an answer"
result = extract_hash_answer(text)  # "Just an answer"
```

### strip\_non\_numeric

```python theme={null}
def strip_non_numeric(text: str) -> str
```

Remove all non-numeric characters except periods.

**Example**:

```python theme={null}
from verifiers.utils.data_utils import strip_non_numeric

text = "The answer is $42.5"
result = strip_non_numeric(text)  # "42.5"
```

## System Prompts

### BOXED\_SYSTEM\_PROMPT

```python theme={null}
BOXED_SYSTEM_PROMPT = "Please reason step by step, and put your final answer within \\boxed{}."
```

Standard prompt for math problems requiring boxed answers.

### THINK\_BOXED\_SYSTEM\_PROMPT

```python theme={null}
THINK_BOXED_SYSTEM_PROMPT = "Think step-by-step inside <think>...</think> tags. Then, give your final answer inside \\boxed{}."
```

Prompt encouraging explicit reasoning in XML tags.

**Example**:

```python theme={null}
import verifiers as vf
from verifiers.utils.data_utils import (
    load_example_dataset,
    format_dataset,
    BOXED_SYSTEM_PROMPT,
)

def load_environment():
    # Load and format dataset
    dataset = load_example_dataset("math", n=100)
    dataset = format_dataset(
        dataset,
        system_prompt=BOXED_SYSTEM_PROMPT,
    )
    
    def correct(answer: str, completion: str, **kwargs) -> float:
        from verifiers.utils.data_utils import extract_boxed_answer
        extracted = extract_boxed_answer(completion)
        return 1.0 if extracted == answer else 0.0
    
    return vf.SingleTurnEnv(
        dataset=dataset,
        rubric=vf.Rubric(correct),
    )
```

## Supported Datasets

<ParamField path="aime2024">
  AIME 2024 math competition (15 problems)
</ParamField>

<ParamField path="aime2025">
  AIME 2025 math competition (30 problems, AIME I + II)
</ParamField>

<ParamField path="amc2023">
  AMC 2023 math competition
</ParamField>

<ParamField path="gpqa_diamond">
  GPQA Diamond subset (high-quality questions)
</ParamField>

<ParamField path="gpqa_main">
  GPQA Main dataset
</ParamField>

<ParamField path="gsm8k">
  Grade School Math 8K dataset
</ParamField>

<ParamField path="math">
  MATH competition dataset
</ParamField>

<ParamField path="math500">
  MATH-500 subset
</ParamField>

<ParamField path="mmlu">
  Massive Multitask Language Understanding
</ParamField>

<ParamField path="mmlu_pro">
  MMLU-Pro (harder variant)
</ParamField>

<ParamField path="openbookqa">
  OpenBookQA question answering
</ParamField>

<ParamField path="openrs / openrs_easy / openrs_hard">
  OpenRS reasoning problems
</ParamField>

<ParamField path="prime_code">
  Prime verifiable coding problems
</ParamField>

## Preprocessing Functions

Internal preprocessing functions used by `load_example_dataset()`:

```python theme={null}
def get_preprocess_fn(name: str) -> Callable[[dict], dict]
```

Returns a preprocessing function for the named dataset. Each preprocessor:

* Extracts `question` and `answer` fields
* Normalizes format (e.g., strips #### delimiters)
* Handles dataset-specific quirks

<Note>
  These are internal functions. Use `load_example_dataset()` instead of calling preprocessors directly.
</Note>

## Custom Dataset Example

```python theme={null}
import verifiers as vf
from verifiers.utils.data_utils import format_dataset
from datasets import Dataset

# Create custom dataset
raw_data = [
    {"question": "What is 2+2?", "answer": "4"},
    {"question": "What is 10*5?", "answer": "50"},
]

dataset = Dataset.from_list(raw_data)

# Format with system prompt
formatted = format_dataset(
    dataset,
    system_prompt="Solve the math problem.",
)

# Use in environment
env = vf.SingleTurnEnv(
    dataset=formatted,
    rubric=vf.Rubric(lambda answer, completion, **kw: 1.0 if answer in completion else 0.0),
)
```

## See Also

* [Environment.make\_dataset()](/api/environment#make_dataset) - Create datasets from dicts
* [HuggingFace Datasets](https://huggingface.co/docs/datasets/) - Dataset library documentation