1
Curious builder0 XP earned · 300 to level 2
0 daysFinish a lesson to begin
Badge collection0 of 6 unlocked
51 small wins to finish your pathNext question →
Retrieval returns the right chunks, but the LLM still gives wrong or hallucinated answers. What do you do?
30-second answerSay your answer out loud first, then reveal.
Symptoms, causes and fixes
Diagnose with traces: look at the exact prompt sent to the model for failing cases.
| Symptom | Likely cause | Fix |
|---|---|---|
| Answer uses facts not in context | Prompt allows parametric knowledge; model "helpful" | Strict grounding instructions; abstention; faithfulness check |
| Correct chunk present but ignored | Too many chunks; buried mid-context | Fewer, reranked chunks; best chunks first |
| Wrong numbers / dates | Misreading tables; arithmetic | Better table formatting; ask for quotes; do calculations in code |
| Mixes two products' details | Similar chunks from different entities | Include entity metadata in chunk headers; filter by entity |
| Answers a different question | Ambiguous query | Clarifying question; query rewriting |
| Outdated answer | Old and new versions both retrieved | Version metadata; prefer latest (Q27, Q32) |
Post-generation verification
- Groundedness check: an NLI model or LLM judge verifies each claim against the context. If unsupported, regenerate or abstain.
- Citation validation: cited chunk IDs exist and contain the claim.
Model choice: larger or newer models are generally better at faithful context use. But first fix context quality, because it's cheaper and helps every model.
Related
This is what real progress feels like.