Updated July 2026

Every LLM feature ships in one of two states: with an eval, or with a vibe. Vibes feel fine right up until a prompt tweak silently breaks a case you fixed last month, and you find out from a user. Some opinions on doing better.

“Looks good to me” doesn’t scale, and it doesn’t even work

Manual spot-checking has two failure modes. The obvious one is volume: you check five outputs and ship, and the failure lives in the sixth shape of input you didn’t think of. The subtle one is drift: when you change a prompt to fix case A, you re-check case A, feel good, and never re-run cases B through F. Regression is the default outcome of prompt iteration without a test set. The entire point of an eval is making the cases you’ve already fixed permanently cheap to re-check.

Twenty real failures beat two hundred synthetic cases

The strongest eval sets are built backwards, from production. Every time the model does something wrong, that input becomes a test case. Twenty of those capture how your traffic actually breaks: the weird formatting, the ambiguous phrasing, the inputs nobody would write while imagining test data. Two hundred generated cases mostly capture what the generating model thinks is hard, which is a different and tidier distribution. Generate cases to fill known gaps, not as the foundation.

Not everything needs an LLM judge

Cheapest first:

  1. Exact checks. Does it parse, does the enum value match, did the right tool get called. Free, deterministic, and a surprising fraction of what matters.
  2. Heuristics. Length bounds, required sections present, forbidden phrases absent, links resolve.
  3. LLM judge. Only for the genuinely fuzzy residue: tone, helpfulness, faithfulness to a source.

A common mistake is reaching for a judge because the task feels fuzzy, when 80% of the quality bar (“did it cite the document,” “is the answer under 100 words,” “is the SQL valid”) is mechanically checkable.

When the judge grades its own homework

LLM judges have known biases: they prefer longer outputs, polished-sounding prose, and answers that resemble what they would have written. Asking a model to grade outputs from the same model family compounds it. Mitigations that work: grade against an explicit rubric with independently checkable criteria rather than “rate 1-10”; ask for the evidence before the verdict; compare pairs instead of scoring in isolation; and periodically audit the judge itself by checking a sample of its grades against your own. A judge you’ve never audited is a random number generator with good vibes. persona-probe, which behavior-tests the personas on this site, takes the compounding problem head-on: it pins the judge to a different model family than the one answering, so the two don’t share the same blind spots.

Make it cheap to run, or it won’t be run

An eval that takes an hour and a notebook gets run before launches. An eval that runs in CI on every prompt change gets run always, and “always” is where the value is. Batch APIs cut the cost in half; a few hundred cases is well inside hobby-budget territory. The bar to clear isn’t building a great eval. It’s making “did this change break anything” a question that answers itself.