Article
Every AI feature we build starts the same way: before any retrieval, prompting, or model choice, we sit with someone who knows the domain and build a test set. Fifty to two hundred real inputs, each with the answer a careful expert would give, and a rule for what counts as a pass. Then we score the current state, which is usually zero, and every change afterwards is measured against it.
This sounds like process for its own sake. It is the opposite. It is the only thing that makes the rest of the work fast.
What the test set changes
The scope. Writing down fifty real questions reveals what the feature actually has to do. About a third of the time, the questions are not what the brief described. Some are lookups that want a database query, not a model. Some are requests to do something, not answer something. We wrote about that pattern in when a RAG system is the wrong answer; the test set is how you find it in week one instead of week eight.
The arguments. "The answers feel worse since the change" is a conversation that goes nowhere. "The score dropped from 81 to 74 and here are the seven cases that regressed" is a fix. Every disagreement about quality becomes a look at specific failures.
The model choice. With a test set, choosing a model is an afternoon: run each, compare scores and costs, pick. Without one, it is a preference. The same goes for a model upgrade six months later, which is the moment most AI features silently get worse.
What a good test set looks like
Real inputs, not invented ones. Real questions are messier than the ones engineers write, and the mess is where features fail. Pull them from support tickets, search logs, or a week of asking users to write down what they wanted to ask.
Graded, not binary, where the task allows it. A summary can be mostly right. Score facts present, facts missing, and facts invented, separately, because they need different fixes.
Small enough to run on every change. Two hundred cases that run in five minutes beat two thousand that run overnight, because the first set gets run.
Growing. Every production failure that reaches a person becomes a test case. After three months the set is the best description of the feature that exists.
Judging with a model, carefully
For open-ended outputs, a second model can grade the first against the reference answer. This works well enough to be useful and badly enough to need checking: we sample the judge's verdicts every week and compare them with a person's. When they drift apart, the judge's prompt gets fixed before anything else does.
What it costs
A week at the start, and an hour a week after that. Against that, it removes the most expensive failure mode in AI work: shipping something that impressed everyone in the demo and nobody in production. Every AI engagement we run has an evaluation phase before the build phase, and the AI Integration workflow shows where it sits in the timeline.