What is LLM evaluation?
LLM evaluation is the process of measuring how well a model — or a prompt, RAG pipeline, or agentic system — performs on a defined task. Good evaluations combine automated metrics, model-graded scoring ("LLM-as-judge"), and human review, and are run regularly to detect regressions.
Also known as: LLM eval, model evaluation, AI benchmarking
Why evals matter more than benchmarks
Public benchmarks (MMLU, HumanEval, MATH) measure general capability and are useful for picking a starting model. But your application's quality depends on YOUR prompts, YOUR data, YOUR users — public benchmarks won't predict that. Custom evaluations on a representative set of your real inputs are the only reliable way to know if a change improved or broke your product.
Building an eval set
Start with 20-50 hand-curated examples covering: typical inputs, edge cases, common failure modes, examples that have caused complaints. Grow over time to 200-500 examples. Each example needs an input and either a reference answer, a grading rubric, or both. Versions matter — track changes to your eval set as carefully as you track changes to your code.
Grading approaches
(1) Exact-match — works for narrow tasks (classification, structured extraction). (2) Reference-based scoring — BLEU, ROUGE, embedding similarity to a reference answer. (3) LLM-as-judge — have a strong model grade outputs against a rubric. Fast and surprisingly reliable for many tasks, but introduces its own biases. (4) Human review — slowest, most reliable, irreplaceable for subjective quality.
Continuous evaluation in production
Don't only eval before launch. Sample production traffic and re-evaluate weekly. Capture user feedback signals (thumbs, edit-after-paste, abandonment) and route low-scoring outputs back into your eval set. Run regressions on every prompt change, model change, or pipeline change. Modern eval platforms (Braintrust, LangSmith, Helicone) automate much of this.