Model Evaluation (Evals)

Anthropic's evals framework — building a custom evaluation suite for your application

# pip install anthropic
from anthropic import Anthropic
import json

client = Anthropic()

# Define your eval cases
eval_cases = [
    {
        "input": "What is LumiChats pricing?",
        "ideal": "₹69 per active day",
        "category": "pricing_accuracy"
    },
    {
        "input": "Can I use LumiChats offline?",
        "ideal": "No — LumiChats requires an internet connection",
        "category": "feature_accuracy"
    },
    {
        "input": "How many AI models does LumiChats support?",
        "ideal": "40+ models",
        "category": "feature_accuracy"
    },
]

def run_eval(case: dict, model: str = "claude-sonnet-4-6") -> dict:
    response = client.messages.create(
        model=model,
        max_tokens=200,
        system="You are a LumiChats support assistant. Answer accurately and concisely.",
        messages=[{"role": "user", "content": case["input"]}]
    )
    output = response.content[0].text

    # Simple string-matching grader (replace with LLM grader for complex tasks)
    passed = case["ideal"].lower() in output.lower()

    return {
        "case": case["input"],
        "ideal": case["ideal"],
        "output": output,
        "passed": passed,
        "category": case["category"],
    }

results = [run_eval(case) for case in eval_cases]
accuracy = sum(r["passed"] for r in results) / len(results)
print(f"Overall accuracy: {accuracy:.1%}")  # → 100.0%

• LLM-as-judge evals: For open-ended tasks where string matching fails, use a strong model (GPT-5.4, Claude Opus) to grade outputs against criteria. Provide a rubric: 'Rate this response 1-5 for accuracy, helpfulness, and conciseness. Return JSON.'
• Regression testing: Run evals on every model change to catch capability regressions — when an improvement in one area degrades another.
• Red team evals: Include adversarial test cases specifically designed to find failure modes, not just measure average performance.
• Human evals for high-stakes decisions: For medical, legal, or financial applications, automated evals should be complemented by domain expert assessment before deployment.

1. What is the difference between capability evals and safety evals for LLMs? (Answer: Capability evals measure what the model CAN do: MMLU (knowledge), HumanEval (coding), GSM8K (math), MT-Bench (instruction following). Safety evals measure what the model does in adversarial or edge-case conditions: does it help with bioweapons? Does it generate CSAM? Does it discriminate? Does it deceive? Safety evals are increasingly required by regulators (UK AI Safety Institute, EU AI Act) and use red-teaming, automated harmful content probes, and consistency testing.)
2. Why is held-out test set integrity critical for model evaluation? (Answer: If any test set examples appear in training data (contamination), the model's score reflects memorisation rather than generalisation. Contaminated scores overestimate real-world performance. Best practices: keep evaluation sets completely private and regenerate them periodically, check training data for n-gram overlaps with test sets, and use multiple private evaluation sets rather than relying solely on public benchmarks.)
3. What is a good strategy for evaluating an LLM on a specific business task (e.g., insurance claim classification)? (Answer: (1) Collect 200–500 real examples with expert-labelled ground truth. (2) Split 80/20 for development/final evaluation. (3) Test multiple models and prompts on development set. (4) Run the final selected system ONCE on the held-out set — never tune on it. (5) Compute precision, recall, and F1 per class. (6) Analyse errors by category to find systematic failure patterns. (7) Set minimum acceptable performance thresholds for each class before deploying.)
4. What is LLM-as-judge evaluation and what are its limitations? (Answer: LLM-as-judge: use GPT-4 or Claude to rate model outputs on quality dimensions (accuracy, clarity, safety) instead of human raters. Scales to millions of evaluations cheaply. Limitations: position bias (prefers first response), verbosity bias (prefers longer responses), self-preference (GPT-4 prefers GPT-4-style responses), cultural bias (reflects RLHF evaluators' cultural background). Mitigation: randomise position, calibrate against human ratings, use multiple judge models.)
5. Why is production monitoring an essential part of model evaluation? (Answer: Offline evaluation on benchmark datasets measures performance on a fixed distribution. Production data is dynamic — user queries evolve, edge cases emerge, the world changes. A model can pass all benchmarks and still fail on real user queries. Production monitoring tracks: user satisfaction signals (thumbs up/down, session length), error rates, harmful content flags, latency, and cost. It closes the loop between evaluation and deployment, enabling continuous improvement.)