Skip to content

What Is Evals (AI Evaluation)? A Plain-English Definition

Evals, short for evaluations, are repeatable tests that score what an AI system produces against success criteria agreed in advance. For an AI agent, they are the measuring stick: without one, nobody can say whether a new model, a reworded prompt or an extra tool made things better or worse.

The OpenAI API documentation describes evals as tests that check model outputs against the style and content criteria you specify, and calls them essential when upgrading or trying new models.

How do evals work?

An eval combines a set of tasks with fixed inputs and success criteria, one or more runs per task, and grading logic. The output is a score, such as the share of tasks passed, that you can compare directly between two versions of a prompt, model or agent.

In a January 2026 engineering post on agent evals, Anthropic breaks this down into a few parts:

  • Task: a single test with defined inputs and success criteria, for example "support ticket about a failed invoice export; expected: correct category and escalation to finance".
  • Trial: one attempt at a task. Because outputs vary between runs, tasks are run several times.
  • Grader: the logic that scores one aspect of performance.
  • Transcript and outcome: for agents, the final answer is not enough. The record of tool calls and the end state of the system being changed matter too.

The Claude documentation lists three grading approaches. Code-based checks are fast and reliable but lack nuance. Human grading is flexible but slow and expensive. Model-based grading is fast and flexible, provided you test its reliability first. The same page recommends many automatically graded cases over a few hand-graded ones.

Picture an agent that triages incoming support tickets. Its eval holds 50 real, anonymised tickets with the correct category and routing, including vague, multilingual and very long ones. Before switching to a new model, every ticket runs several times; the change ships only if the pass rate holds.

"Evals" is the umbrella term. Three related terms describe parts of it:

  • Golden dataset: the vetted collection of test cases and correct answers that an eval runs on.
  • LLM-as-a-judge: one grading approach, in which a language model scores outputs against a rubric.
  • Regression testing: one purpose of evals. Anthropic separates capability evals, which start with a low pass rate and track new skills, from regression evals, which should pass almost completely and flag anything that breaks.

What to watch out for

An eval only measures what its test cases contain. If it lacks empty inputs, oversized documents or ambiguous requests, it will report success while the agent fails in production. Hallucinations likewise surface only when cases are designed to provoke them.

  • Build cases from real work and real failures, not idealised examples.
  • Check model-based grading against human judgement on a sample before trusting it.
  • Start early: Anthropic notes that evals get harder to build the longer you wait.

If you run AI agents in your business, treat an eval run as the gate for every model or prompt change. Replacing the underlying large language model is a typical trigger.

Frequently asked questions

How big does a first eval need to be?

Anthropic suggests 20 to 50 simple tasks drawn from real failures as a good starting point. Coverage of your real workload and its awkward cases matters more than the raw count.

Can I run each test case once?

Usually not. Model outputs vary between runs, so running each task several times shows whether an agent is reliable or only right some of the time.

Do evals replace human review?

No. Domain experts still define what good looks like and spot-check whether automated grading agrees with them. Evals make that judgement repeatable at scale.

Do I need a dedicated platform?

Not to start. A script plus a spreadsheet of inputs and expected results is a valid eval. Model providers such as OpenAI also offer their own eval tooling.

Sources

  1. Anthropic Engineering: Demystifying evals for AI agents (9 January 2026) anthropic.com
  2. Claude documentation: Define success criteria and build evaluations (accessed October 2026) platform.claude.com
  3. OpenAI API documentation: Evals (accessed October 2026) developers.openai.com