Golden datasets: knowing an AI system works
A golden dataset is a small collection of real inputs, each with the behaviour you expect. Run the system on it before every release and you know whether a change made things better or worse.
Why a small set beats a big guess
Trying a few questions by hand feels like testing, but it changes every time and nobody writes it down. Thirty carefully chosen cases, run the same way every time, give you a number you can compare across releases.
What goes in it
| Kind of case | Why it is there |
|---|---|
| Common requests | The bulk of real traffic; the system must handle these well |
| Edge cases | Unusual but valid inputs that break naive systems |
| Must refuse | Questions with no answer in the data, or that the user may not ask |
| Adversarial | Inputs that try to make the system ignore its instructions |
Take cases from real data wherever possible: support tickets, search logs or questions users actually asked. Invented examples tend to be easier than the real thing.
Expected behaviour, not exact wording
A model rarely produces the same sentence twice, so compare behaviour rather than text. Did it pick the right category? Did it cite the right document? Did it refuse when it should?
- Exact checks for things that must match: a category, a number, a tool that must be called.
- Rule checks for things you can test in code: a citation is present, no personal data appears.
- A judging model for open answers, scored against a written rubric, and checked against human labels before you trust it.
Agree the threshold
Keep it honest
- Add every real failure you find in production as a new case
- Keep some cases aside and never tune against them, so the set cannot be memorised
- Record the score for every release so you can see the trend
- Thirty real cases run the same way beat a hundred tries by hand
- Check behaviour, not exact wording
- The threshold is agreed, not invented
- Write ten cases for a system you have built: six common, two edge, two that must be refused
- Decide how each one will be checked: exact, rule or judge
Every expert started right here.