> ## Documentation Index
> Fetch the complete documentation index at: https://mintlify.com/primeintellect-ai/verifiers/llms.txt
> Use this file to discover all available pages before exploring further.

# Wordle Game Environment

> Train models to play Wordle using multi-turn interaction and feedback

This example demonstrates a multi-turn game environment where models play Wordle by making guesses and receiving feedback. It showcases parsing structured output, custom reward functions, and integration with TextArena.

## Overview

The Wordle environment provides:

* **Game**: 5-letter word guessing with color-coded feedback
* **Format**: Multi-turn interaction (up to 6 guesses)
* **Parsing**: XML tags for structured guess extraction
* **Rewards**: Correctness, efficiency bonus, and partial credit
* **Integration**: TextArena game library

## Complete Implementation

Here's the full working implementation from `environments/wordle/wordle.py`:

```python theme={null}
import re

import verifiers as vf
from verifiers.envs.integrations.textarena_env import TextArenaEnv

DEFAULT_SYSTEM_PROMPT = """You are a competitive game player. \
Make sure you read the game instructions carefully, and always follow the required format.

In each turn, think step-by-step, then give your guess inside <guess>...</guess> tags."""


### feedback functions
def wordle_feedback_fn(observation: str) -> str:
    latest_observation = observation.split("[GAME]")[-1].strip()
    if "Feedback:" in latest_observation:
        return latest_observation.split("Feedback:")[-1]
    else:
        return latest_observation


### reward functions
def correct_answer(parser, completion, answer, **kwargs) -> float:
    """Whether the guess is *exactly* correct."""
    guess = parser.parse_answer(completion)
    return 1.0 if guess == "[" + answer + "]" else 0.0


def length_bonus(parser, completion, answer, **kwargs) -> float:
    """Bonus for shorter correct solutions."""
    assistant_messages = parser.get_assistant_messages(completion)
    guesses = [
        x for x in assistant_messages if re.search(r"<guess>.*</guess>", x["content"])
    ]
    is_correct = correct_answer(parser, completion, answer, **kwargs)
    return is_correct / (len(guesses) or 1)


def partial_answer(parser, completion, answer, **kwargs) -> float:
    """Partial credit for the latest guess."""
    if correct_answer(parser, completion, answer, **kwargs):
        return 0.0
    user_messages = parser.get_user_messages(completion)
    for user_message in user_messages[::-1]:
        feedback = user_message["content"].strip()
        feedback_parts = feedback.split("\n")
        if len(feedback_parts) == 3:
            _, scoring, _ = feedback_parts
            scoring = scoring.strip()
            num_greens = scoring.count("G")
            num_yellows = scoring.count("Y")
            return 0.2 * num_greens + 0.1 * num_yellows
    return 0.0


### environment loader
def load_environment(
    num_train_examples: int = 2000,
    num_eval_examples: int = 20,
    system_prompt: str = DEFAULT_SYSTEM_PROMPT,
    seed: int = 0,
    **kwargs,
):
    parser = vf.XMLParser(fields=["guess"], answer_field="guess")

    rubric = vf.Rubric(parser=parser)
    rubric.add_reward_func(correct_answer)
    rubric.add_reward_func(partial_answer)
    rubric.add_reward_func(length_bonus)
    format_reward = parser.get_format_reward_func()
    format_reward.__name__ = "format_reward"
    rubric.add_reward_func(format_reward, weight=0.2)

    return TextArenaEnv(
        game="Wordle-v0",
        num_train_examples=num_train_examples,
        num_eval_examples=num_eval_examples,
        feedback_fn=wordle_feedback_fn,
        seed=seed,
        system_prompt=system_prompt,
        parser=parser,
        rubric=rubric,
        **kwargs,
    )
```

## How It Works

### 1. TextArena Integration

`TextArenaEnv` wraps TextArena games for RL training:

```python theme={null}
return TextArenaEnv(
    game="Wordle-v0",
    num_train_examples=2000,
    num_eval_examples=20,
    feedback_fn=wordle_feedback_fn,
)
```

**TextArena** provides text-based game environments. The Wordle game:

* Generates random 5-letter target words
* Accepts guesses and returns color-coded feedback
* Tracks game state (remaining guesses, history)

### 2. Structured Output Parsing

`XMLParser` extracts guesses from model responses:

```python theme={null}
parser = vf.XMLParser(fields=["guess"], answer_field="guess")
```

**Example parsing**:

```python theme={null}
model_output = "I'll try the word CRANE. <guess>CRANE</guess>"
parser.parse_answer(model_output)  # Returns: "[CRANE]"
```

### 3. Feedback Processing

The `wordle_feedback_fn` extracts game feedback:

```python theme={null}
def wordle_feedback_fn(observation: str) -> str:
    latest_observation = observation.split("[GAME]")[-1].strip()
    if "Feedback:" in latest_observation:
        return latest_observation.split("Feedback:")[-1]
    else:
        return latest_observation
```

**Example feedback**:

```
Guess: CRANE
Feedback: [Y][G][_][_][Y]
Remaining guesses: 5
```

**Legend**:

* `G` = Green (correct letter, correct position)
* `Y` = Yellow (correct letter, wrong position)
* `_` = Gray (letter not in word)

### 4. Multi-Part Reward Function

Three reward components encourage different behaviors:

<Tabs>
  <Tab title="Correctness">
    ```python theme={null}
    def correct_answer(parser, completion, answer, **kwargs) -> float:
        """Whether the guess is *exactly* correct."""
        guess = parser.parse_answer(completion)
        return 1.0 if guess == "[" + answer + "]" else 0.0
    ```

    **Returns**: 1.0 for correct word, 0.0 otherwise
  </Tab>

  <Tab title="Efficiency Bonus">
    ```python theme={null}
    def length_bonus(parser, completion, answer, **kwargs) -> float:
        """Bonus for shorter correct solutions."""
        assistant_messages = parser.get_assistant_messages(completion)
        guesses = [
            x for x in assistant_messages 
            if re.search(r"<guess>.*</guess>", x["content"])
        ]
        is_correct = correct_answer(parser, completion, answer, **kwargs)
        return is_correct / (len(guesses) or 1)
    ```

    **Returns**:

    * 1st guess: 1.0 bonus
    * 2nd guess: 0.5 bonus
    * 3rd guess: 0.33 bonus
    * Wrong: 0.0
  </Tab>

  <Tab title="Partial Credit">
    ```python theme={null}
    def partial_answer(parser, completion, answer, **kwargs) -> float:
        """Partial credit for the latest guess."""
        if correct_answer(parser, completion, answer, **kwargs):
            return 0.0  # Already got full credit
        
        # Find latest feedback
        user_messages = parser.get_user_messages(completion)
        for user_message in user_messages[::-1]:
            feedback = user_message["content"].strip()
            feedback_parts = feedback.split("\n")
            if len(feedback_parts) == 3:
                _, scoring, _ = feedback_parts
                scoring = scoring.strip()
                num_greens = scoring.count("G")
                num_yellows = scoring.count("Y")
                return 0.2 * num_greens + 0.1 * num_yellows
        return 0.0
    ```

    **Returns**:

    * 0.2 per green letter
    * 0.1 per yellow letter
    * Max: 1.0 (all green = correct word)
  </Tab>

  <Tab title="Format Compliance">
    ```python theme={null}
    format_reward = parser.get_format_reward_func()
    rubric.add_reward_func(format_reward, weight=0.2)
    ```

    **Returns**: 1.0 if output contains `<guess>...</guess>`, 0.0 otherwise

    Weighted at 0.2 to encourage proper formatting without dominating the reward.
  </Tab>
</Tabs>

### Combined Reward

```python theme={null}
rubric = vf.Rubric(parser=parser)
rubric.add_reward_func(correct_answer, weight=1.0)      # Main reward
rubric.add_reward_func(partial_answer, weight=1.0)      # Progress signal
rubric.add_reward_func(length_bonus, weight=1.0)        # Efficiency
rubric.add_reward_func(format_reward, weight=0.2)       # Format compliance
```

**Total reward** = weighted sum of all components

## Example Game Trajectory

<Tabs>
  <Tab title="Successful Game">
    **Target word**: STEAK

    ***

    **Turn 1**

    **Assistant**: I'll start with a common word to test vowels and common consonants.
    <guess>CRANE</guess>

    **Environment**:

    ```
    Guess: CRANE
    Feedback: [_][_][Y][_][Y]
    Remaining guesses: 5
    ```

    ***

    **Turn 2**

    **Assistant**: A is in position 3 (wrong spot), E is in position 5 (wrong spot). Let me try:
    <guess>STEAL</guess>

    **Environment**:

    ```
    Guess: STEAL
    Feedback: [G][G][G][G][_]
    Remaining guesses: 4
    ```

    ***

    **Turn 3**

    **Assistant**: Almost there! The first four letters are S-T-E-A. Last letter isn't L. Let me try:
    <guess>STEAK</guess>

    **Environment**:

    ```
    Guess: STEAK
    Feedback: [G][G][G][G][G]
    You won! The word was STEAK.
    ```

    **Rewards**:

    * `correct_answer`: 1.0
    * `length_bonus`: 1.0 / 3 = 0.33
    * `partial_answer`: 0.0 (already correct)
    * `format_reward`: 1.0
    * **Total**: 2.53
  </Tab>

  <Tab title="Failed Game">
    **Target word**: QUIRK

    ***

    **Turn 1-6**: Model makes guesses but never finds QUIRK

    **Turn 7**

    **Environment**:

    ```
    Game over! You've used all 6 guesses.
    The word was QUIRK.
    ```

    **Rewards** (from last guess with feedback `[_][Y][_][G][_]`):

    * `correct_answer`: 0.0
    * `length_bonus`: 0.0 (not correct)
    * `partial_answer`: 0.2 \* 1 + 0.1 \* 1 = 0.3
    * `format_reward`: 1.0
    * **Total**: 0.5
  </Tab>

  <Tab title="Format Error">
    **Turn 1**

    **Assistant**: I'll try HOUSE

    **Environment**: *(No valid guess extracted)*

    **Rewards**:

    * `correct_answer`: 0.0
    * `length_bonus`: 0.0
    * `partial_answer`: 0.0
    * `format_reward`: 0.0 (missing guess tags)
    * **Total**: 0.0

    *The model failed to use the required format and received no reward.*
  </Tab>
</Tabs>

## Running the Environment

### Installation

```bash theme={null}
# Install with TextArena integration
prime env install wordle
```

### Quick Evaluation

```bash theme={null}
# Evaluate on 20 games
prime eval run wordle \
  -m openai/gpt-4.1-mini \
  -b https://api.openai.com/v1 \
  -k OPENAI_API_KEY \
  -n 20 \
  -r 5
```

### Training Dataset

```bash theme={null}
# Generate training data with 1000 games
prime eval run wordle \
  -m openai/gpt-4.1-mini \
  -a '{"num_train_examples": 1000, "num_eval_examples": 50}' \
  --split train \
  -n 1000 \
  -r 8
```

## Configuration Options

| Parameter            | Default                 | Description                     |
| -------------------- | ----------------------- | ------------------------------- |
| `num_train_examples` | `2000`                  | Number of training games        |
| `num_eval_examples`  | `20`                    | Number of evaluation games      |
| `system_prompt`      | `DEFAULT_SYSTEM_PROMPT` | Instructions for the model      |
| `seed`               | `0`                     | Random seed for word generation |

## Key Features

### Structured Output with XMLParser

`XMLParser` provides:

* **Extraction**: Pulls content from XML tags
* **Validation**: Checks format compliance
* **Format rewards**: Built-in reward function for proper formatting

```python theme={null}
parser = vf.XMLParser(
    fields=["guess"],           # Fields to extract
    answer_field="guess"        # Which field is the answer
)
```

### Multi-Component Rewards

Combining multiple reward signals:

* **Sparse signal** (`correct_answer`): Only 1.0 when winning
* **Dense signal** (`partial_answer`): Credit for progress
* **Efficiency** (`length_bonus`): Reward faster solutions
* **Compliance** (`format_reward`): Enforce output format

This creates a rich learning signal for RL training.

### Game State Tracking

TextArenaEnv automatically tracks:

* Number of guesses made
* Guess history
* Remaining attempts
* Win/loss status

## Metrics Tracked

* `correct_answer`: 1.0 if word guessed correctly
* `length_bonus`: Efficiency bonus (0.0 to 1.0)
* `partial_answer`: Progress score (0.0 to 1.0)
* `format_reward`: Format compliance (0.0 or 1.0)
* `reward`: Combined weighted sum
* `num_turns`: Number of guesses made

## Advanced Usage

### Custom Reward Weights

Adjust the importance of different reward components:

```python theme={null}
rubric = vf.Rubric(parser=parser)
# Emphasize correctness over efficiency
rubric.add_reward_func(correct_answer, weight=5.0)     # 5x weight
rubric.add_reward_func(partial_answer, weight=2.0)     # 2x weight
rubric.add_reward_func(length_bonus, weight=0.5)       # 0.5x weight
rubric.add_reward_func(format_reward, weight=0.2)      # 0.2x weight
```

### Different Wordle Variants

TextArena supports multiple Wordle variants:

```python theme={null}
from verifiers.envs.integrations.textarena_env import TextArenaEnv

# 6-letter Wordle
env_6 = TextArenaEnv(game="Wordle-6-v0", ...)

# 7-letter Wordle
env_7 = TextArenaEnv(game="Wordle-7-v0", ...)
```

### Other TextArena Games

The same pattern works for other TextArena games:

```python theme={null}
# Hangman
env = TextArenaEnv(
    game="Hangman-v0",
    parser=vf.XMLParser(fields=["guess"]),
    ...
)

# 20 Questions
env = TextArenaEnv(
    game="TwentyQuestions-v0",
    parser=vf.XMLParser(fields=["question"]),
    ...
)
```

## Related Examples

* [GSM8K](/examples/gsm8k) - Single-turn reasoning
* [Math Python](/examples/math-python) - Multi-turn with code execution
* [Wiki Search](/examples/wiki-search) - Multi-turn with custom tools

## Next Steps

* Learn about [MultiTurnEnv](/essentials/environments#multi-turn) for game environments
* See [Parsers](/essentials/parsers) for structured output extraction
* Explore [Rubrics](/essentials/rubrics) for custom reward design
* Check out [TextArena](https://github.com/LeonGuertler/TextArena) for more games