> ## Documentation Index
> Fetch the complete documentation index at: https://mintlify.com/primeintellect-ai/verifiers/llms.txt
> Use this file to discover all available pages before exploring further.

# Rubric

> Base class for reward functions in RL environments

## Overview

The `Rubric` class is the foundation for evaluating LLM responses in Verifiers environments. It manages reward functions and their weights, supports both individual and group-level scoring, and integrates with parsers to extract answers from completions.

## Constructor

```python theme={null}
Rubric(
    funcs: list[RewardFunc | GroupRewardFunc] | None = None,
    weights: list[float] | None = None,
    parser: vf.Parser | None = None,
)
```

<ParamField path="funcs" type="list[RewardFunc | GroupRewardFunc] | None" default="None">
  List of reward functions to evaluate. Can be individual-level (`RewardFunc`) or group-level (`GroupRewardFunc`) functions.
</ParamField>

<ParamField path="weights" type="list[float] | None" default="None">
  Weights for each reward function. Must match the length of `funcs`. Defaults to 1.0 for each function if not provided.
</ParamField>

<ParamField path="parser" type="vf.Parser | None" default="None">
  Parser instance for extracting answers from completions. Defaults to `vf.Parser()` if not provided.
</ParamField>

## Reward Function Signatures

### Individual-level RewardFunc

Reward functions that score single rollouts can accept any combination of:

* `prompt`: `list[dict[str, str]] | str` - The input prompt
* `completion`: `list[dict[str, str]] | str` - The model's completion
* `answer`: `Any` - Ground truth or metadata for scoring
* `task`: `str` - Task type identifier
* `state`: `State` - Full state dictionary
* `info`: `dict` - Additional metadata
* `**kwargs` - Additional keyword arguments

**Returns**: `float`

### Group-level GroupRewardFunc

Reward functions that score multiple rollouts together accept plural parameters:

* `prompts`: `list[...]` - List of prompts
* `completions`: `list[...]` - List of completions
* `answers`: `list[...]` - List of answers
* `tasks`: `list[str]` - List of task types
* `states`: `list[State]` - List of states
* `infos`: `list[dict]` - List of metadata

**Returns**: `list[float]`

## Methods

### add\_reward\_func

```python theme={null}
def add_reward_func(self, func: RewardFunc, weight: float = 1.0)
```

Add a reward function that contributes to the total reward.

<ParamField path="func" type="RewardFunc">
  The reward function to add.
</ParamField>

<ParamField path="weight" type="float" default="1.0">
  Weight for this function in the total reward calculation.
</ParamField>

### add\_metric

```python theme={null}
def add_metric(self, func: RewardFunc, weight: float = 0.0)
```

Add a metric function that is tracked but doesn't contribute to reward (weight = 0).

<ParamField path="func" type="RewardFunc">
  The metric function to add.
</ParamField>

<ParamField path="weight" type="float" default="0.0">
  Weight for this function (typically 0 for metrics).
</ParamField>

### add\_class\_object

```python theme={null}
def add_class_object(self, name: str, obj: Any)
```

Register a class object that will be passed to reward functions as a keyword argument.

<ParamField path="name" type="str">
  The parameter name that reward functions can use to access this object.
</ParamField>

<ParamField path="obj" type="Any">
  The object to make available to reward functions.
</ParamField>

### score\_rollout

```python theme={null}
async def score_rollout(self, state: State)
```

Evaluate all individual-level reward functions for a single rollout. Updates `state["reward"]` and `state["metrics"]` in place.

<ParamField path="state" type="State">
  The state dictionary to score. Must contain `prompt`, `completion`, and other required fields.
</ParamField>

<Note>
  This method requires at least one individual-level reward function and no group-level functions.
</Note>

### score\_group

```python theme={null}
async def score_group(self, states: list[State])
```

Score multiple rollouts together. Executes all reward functions (both individual and group-level) and updates each state's `reward`, `advantage`, and `metrics` fields.

<ParamField path="states" type="list[State]">
  List of state dictionaries to score together.
</ParamField>

<Note>
  Group-level functions see all states at once and can implement comparative scoring strategies.
</Note>

## Attributes

<ParamField path="funcs" type="list[RewardFunc | GroupRewardFunc]">
  List of registered reward functions.
</ParamField>

<ParamField path="weights" type="list[float]">
  Weights corresponding to each function.
</ParamField>

<ParamField path="parser" type="vf.Parser">
  Parser instance for extracting answers.
</ParamField>

<ParamField path="class_objects" type="dict[str, Any]">
  Dictionary of objects available to reward functions, including the parser.
</ParamField>

## Example Usage

```python theme={null}
import verifiers as vf

# Define custom reward functions
def length_reward(completion, **kwargs):
    """Reward longer responses."""
    text = completion if isinstance(completion, str) else completion[-1]["content"]
    return min(len(text) / 1000, 1.0)

def correctness_reward(completion, answer, parser, **kwargs):
    """Check if parsed answer matches expected."""
    parsed = parser.parse_answer(completion)
    return 1.0 if parsed == answer else 0.0

# Create rubric with weighted functions
rubric = vf.Rubric(
    funcs=[correctness_reward, length_reward],
    weights=[1.0, 0.1],  # Correctness weighted 10x more than length
    parser=vf.Parser()
)

# Add a metric that doesn't affect reward
rubric.add_metric(lambda completion, **kw: len(completion), weight=0.0)

# Score a state
state = {
    "prompt": "What is 2+2?",
    "completion": [{"role": "assistant", "content": "4"}],
    "answer": "4",
    "task": "math",
    "timing": {"scoring_ms": 0, "total_ms": 0}
}

await rubric.score_rollout(state)
print(f"Reward: {state['reward']}")  # 1.0 * 1.0 + 0.001 * 0.1 = 1.0001
print(f"Metrics: {state['metrics']}")  # Individual scores
```

## Group Scoring Example

```python theme={null}
def relative_quality(completions, **kwargs):
    """Group function: reward top 50% of responses."""
    lengths = [len(c[-1]["content"]) for c in completions]
    median = sorted(lengths)[len(lengths) // 2]
    return [1.0 if l >= median else 0.0 for l in lengths]

rubric = vf.Rubric(
    funcs=[relative_quality],
    weights=[1.0]
)

# Score multiple states together
states = [create_state(i) for i in range(10)]
await rubric.score_group(states)

# Each state now has reward, advantage, and metrics
for state in states:
    print(f"Reward: {state['reward']}, Advantage: {state['advantage']}")
```

## See Also

* [JudgeRubric](/api/judge-rubric) - LLM-as-judge scoring
* [MathRubric](/api/math-rubric) - Mathematical equivalence checking
* [Parser](/api/parser) - Base parser class