How do you generate useful synthetic test data for evals, and what are the pitfalls?

30-second answerSay your answer out loud first, then reveal.

Generate in a structured way rather than "write 100 test questions". Define dimensions (user personas, intents/features, scenarios, difficulty, language, edge conditions), sample combinations, have an LLM write realistic inputs for each combination (optionally grounded in real documents or data), then filter, deduplicate, and have humans review a sample. Use synthetic data to cover gaps in real data, especially rare and adversarial cases, not to replace it.

A synthetic test data pipeline: define dimensions, sample combinations, have an LLM write grounded inputs, filter, review a sample by hand, label expected outcomes, and store them as an eval dataset tagged as synthetic.

Example dimensions for a banking assistant

Persona: first-time app user, senior citizen, small business owner, frustrated customer.
Intent: card block, failed UPI transaction, loan eligibility, KYC update.
Scenario: missing information, wrong account mentioned, multiple issues at once.
Language: English, Hindi, Hinglish (typed in Latin script).

Pitfalls

Pitfall	Mitigation
Too clean / too similar to the docs' wording (easy retrieval)	Instruct the generator to paraphrase like real users, with typos and slang; compare with real logs
Low diversity (mode collapse)	Sample dimensions explicitly; vary seeds and models; dedupe by embedding
Generator bias = model-under-test bias	Use a different model family to generate; human review
Wrong expected answers	Expert verification of labels
Over-reliance	Keep real data as the backbone; tag synthetic examples to analyse them separately

PreviousHow do you make eval results statistically trustworthy?

Every expert started right here.

How do you generate useful synthetic test data for evals, and what are the pitfalls?

Example dimensions for a banking assistant

Pitfalls

Related