Config Utilities

Environment variable validation utilities

Config Utilities

Utilities for validating required environment variables, particularly API keys.

Overview

The verifiers.utils.config_utils module provides:

• Environment variable validation
• Clear error messages for missing keys
• Guidance on setting secrets in different contexts

Functions

ensure_keys

def ensure_keys(keys: list[str]) -> None

Validate that required environment variables are set.

keys · list[str] · required

List of environment variable names to check (e.g., ["OPENAI_API_KEY", "ANTHROPIC_API_KEY"]).

Raises: MissingKeyError if any keys are not set in the environment.

Example:

import verifiers as vf

def load_environment():
    # Validate required API keys
    vf.ensure_keys(["OPENAI_API_KEY"])
    
    return vf.SingleTurnEnv(
        dataset=dataset,
        rubric=rubric,
    )

Exceptions

MissingKeyError

class MissingKeyError(ValueError):
    def __init__(self, keys: list[str])

Exception raised when required environment variables are missing. Provides helpful error messages with context-specific guidance for setting environment variables in different deployment scenarios.

Attributes:

keys · list[str]

List of missing environment variable names.

Error Message Format:

Missing required environment variable(s): OPENAI_API_KEY, ANTHROPIC_API_KEY

To set these variables:
  - Environments Hub CI: Add secrets on the environment's Settings page
  - Hosted Training: Set env_file in your config (e.g. env_file = ["secrets.env"])
  - Local: Export in your shell (e.g. export OPENAI_API_KEY=...)

Example:

import verifiers as vf

try:
    vf.ensure_keys(["OPENAI_API_KEY", "ANTHROPIC_API_KEY"])
except vf.MissingKeyError as e:
    print(f"Missing {len(e.keys)} keys: {e.keys}")
    # Missing 2 keys: ['OPENAI_API_KEY', 'ANTHROPIC_API_KEY']
    raise

Example Usage

Basic Validation

import verifiers as vf
from datasets import load_dataset

def load_environment():
    # Ensure OpenAI key is set
    vf.ensure_keys(["OPENAI_API_KEY"])
    
    dataset = load_dataset("gsm8k", "main", split="test")
    
    return vf.SingleTurnEnv(
        dataset=dataset,
        rubric=vf.Rubric(lambda **kw: 1.0),
    )

Multiple Keys

import verifiers as vf

def load_environment():
    # Check multiple API keys
    vf.ensure_keys([
        "OPENAI_API_KEY",
        "ANTHROPIC_API_KEY",
        "BROWSERBASE_API_KEY",
    ])
    
    # ... environment setup ...

Custom Services

import verifiers as vf

def load_environment():
    # Validate custom service keys
    vf.ensure_keys([
        "GITHUB_TOKEN",
        "SLACK_WEBHOOK_URL",
        "DATABASE_URL",
    ])
    
    # ... use these in your environment ...

Error Handling

import verifiers as vf

def load_environment():
    try:
        vf.ensure_keys(["OPENAI_API_KEY"])
    except vf.MissingKeyError as e:
        print(f"Missing keys: {', '.join(e.keys)}")
        print("Please set environment variables before continuing.")
        raise
    
    # Safe to use API key here
    return create_environment()

Setting Environment Variables

Local Development

# Export in shell
export OPENAI_API_KEY=sk-...
export ANTHROPIC_API_KEY=sk-ant-...

# Or use .env file with python-dotenv
pip install python-dotenv

# load_dotenv in your script
from dotenv import load_dotenv
load_dotenv()

import verifiers as vf
vf.ensure_keys(["OPENAI_API_KEY"])  # Now works

Environments Hub CI

1. Go to your environment’s page on Environments Hub
2. Click Settings
3. Add secrets in the Environment Variables section
4. Secrets are automatically injected during CI runs

Hosted Training

Create a secrets.env file:

# secrets.env
OPENAI_API_KEY=sk-...
ANTHROPIC_API_KEY=sk-ant-...

Reference in your training config:

# config.py
from verifiers.trainer import TrainConfig

config = TrainConfig(
    env_id="my_env",
    env_file=["secrets.env"],  # Load secrets
    # ... other config ...
)

Best Practices

1. Validate Early

Call ensure_keys() in load_environment() to fail fast:

def load_environment():
    # Fail immediately if keys missing
    vf.ensure_keys(["OPENAI_API_KEY"])
    
    # Expensive operations only run if keys present
    dataset = expensive_data_loading()
    return create_env(dataset)

2. Document Required Keys

List required keys in your environment’s README:

# My Environment

## Required Environment Variables

- `OPENAI_API_KEY`: OpenAI API key for model inference
- `BROWSERBASE_API_KEY`: Browserbase API key for browser automation

## Setup

```bash
export OPENAI_API_KEY=sk-...
export BROWSERBASE_API_KEY=...

### 3. Validate Only What You Use

Don't validate keys you don't need:

```python
# Bad: Requires unnecessary keys
vf.ensure_keys(["OPENAI_API_KEY", "ANTHROPIC_API_KEY"])
# ... but only uses OpenAI

# Good: Only check what's needed
vf.ensure_keys(["OPENAI_API_KEY"])

4. Use Consistent Key Names

Follow common conventions:

• *_API_KEY for API keys
• *_URL for endpoints
• *_TOKEN for auth tokens

vf.ensure_keys([
    "OPENAI_API_KEY",      # API key
    "DATABASE_URL",        # Connection string
    "GITHUB_TOKEN",        # Auth token
])

Common API Key Names

OPENAI_API_KEY

OpenAI API key (starts with sk-)

ANTHROPIC_API_KEY

Anthropic API key (starts with sk-ant-)

PRIME_API_KEY

Prime API key for inference

BROWSERBASE_API_KEY

Browserbase API key for browser environments

GITHUB_TOKEN

GitHub personal access token

HF_TOKEN

HuggingFace access token

Error Message Customization

The default error message provides context-specific guidance. You can catch and customize:

import verifiers as vf

def load_environment():
    try:
        vf.ensure_keys(["CUSTOM_API_KEY"])
    except vf.MissingKeyError as e:
        print("\nCustom instructions:")
        print("Get your API key from: https://custom-service.com/api-keys")
        print("Then set: export CUSTOM_API_KEY=<your-key>\n")
        raise

See Also

• Environment - Base environment class
• ClientConfig - Configuring API clients