Evaluation
Evaluation systematically measures the quality of a model's or system's outputs against defined criteria.
Why Evaluation Matters
Reading a handful of outputs and deciding they "look good" doesn't scale, and it doesn't catch regressions when a prompt, model version, or pipeline changes. Evaluation replaces that gut check with a systematic, repeatable process for measuring quality against defined criteria — so you know whether a change actually made things better or worse, not just whether it feels different.
Key Idea
Evaluation isn't a one-time report — it's a process you rerun every time something in the system changes, so you catch regressions before users do.
What Gets Evaluated
- Task-specific quality — does the output actually accomplish what it's meant to (a correct classification, an accurate summary, a working piece of code)?
- Format compliance — for structured output, does the response actually match the expected schema?
- Safety and policy — does the output avoid disallowed content or behavior?
- Consistency — does the system behave predictably across similar inputs, not wildly differently on near-identical requests?
- System-specific dimensions — for a RAG system this includes groundedness and retrieval relevance; for an agent it includes whether it selected the right tools and reached the goal.
How Evaluation Is Run
- Golden test sets — representative inputs with known-good expected outputs, run automatically whenever something changes.
- LLM-as-judge — using a separate model call to score outputs against criteria, which scales far better than manual review alone.
- Human review — periodic spot-checking, especially valuable for catching failure modes an automated judge might share or miss.
- A/B or before/after comparison — comparing a proposed change against the current system on the same test set before rolling it out.
Warning
An unrepresentative test set gives false confidence — it can pass cleanly while missing exactly the failure modes real users encounter.
Common Mistakes
Evaluating once and never again
Changes to prompts, models, or pipelines can silently regress quality — evaluation needs to be repeatable, not a one-time report.
Using an unrepresentative test set
A test set that doesn't reflect real usage can look great while missing the failures that actually matter in production.
Trusting LLM-as-judge without any human review
An automated judge can share blind spots with the system it's evaluating — periodic human review catches what automation misses.
Evaluating only the final output
For multi-stage systems like RAG or agents, evaluating only the end result can hide which specific stage is actually failing.
Treating a passing evaluation as a permanent guarantee
A system that evaluates well today can regress after any change — evaluation needs to run continuously, not just once at launch.
Interview Question
How would you build an evaluation process for an AI system, and why is it necessary?
Evaluation systematically measures output quality against defined criteria, because reading a handful of examples and deciding they look fine doesn't scale and doesn't catch regressions when something changes. I'd build a representative golden test set with known-good expected outputs, use LLM-as-judge to score outputs against criteria at scale, and still do periodic human review to catch what an automated judge might share as a blind spot. Critically, I'd rerun that evaluation every time the prompt, model, or pipeline changes, since a system that evaluates well today can regress after any change — evaluation is a repeatable process, not a one-time report.
What an interviewer may ask next
- Why is an unrepresentative test set worse than having no formal evaluation at all?
- Why would you still want human review even with a working LLM-as-judge setup?
- How does evaluating a multi-stage system, like RAG or an agent, differ from evaluating a single model call?
Explain It in 30 Seconds
Evaluation systematically measures whether a model's or system's outputs meet defined criteria, replacing a gut-check with something repeatable — so you can tell whether a change actually helped or hurt. It typically combines a representative golden test set, LLM-as-judge scoring to scale beyond manual review, and periodic human review to catch what automation misses. It needs to rerun every time something changes, since a system that evaluates well today can regress after any update.