You're building RAG for a legal or medical setting where a wrong answer is very costly. How do you design for maximum trustworthiness?

Design principles

1. Source control: only approved, versioned documents (current guidelines, statutes, internal protocols), with effective dates and jurisdiction metadata. No random web content.
2. Retrieval precision: hybrid + reranking, domain-tuned embeddings, jurisdiction and version filters, exact-match support for citations ("Section 138 NI Act").
3. Constrained generation:
   • Extract verbatim quotes first, then answer only from the quotes.
   • Every sentence carries a citation linking to the exact passage.
   • Structured output: answer, quotes, sources, confidence, caveats.
4. Verification layer:
   • An NLI model or LLM judge checks that each claim is entailed by its cited quote.
   • Programmatic check that the quotes exist verbatim in the source.
   • Fail → abstain or route to a human.
5. Abstention and escalation: below a confidence threshold, or when sources conflict, say so and escalate. Tune for low false-positive answers even at the cost of more "I don't know" responses.
6. Human in the loop: the professional reviews before acting. The UI highlights the source passages for fast verification.
7. Evaluation: expert-labelled test sets; track critical-error rate separately from average quality; adversarial and edge cases; regression gates before every change.
8. Compliance: audit trails, data privacy (patient or client data), disclaimers, and incident processes.

Key metric: not "accuracy" but the rate of confidently wrong answers, which must approach zero.

Related

• Previous: Q39. What is GraphRAG? When would you use a knowledge graph with RAG?
• Next: Q41. An attacker uploads a document containing hidden instructions that get retrieved into the prompt. What are the risks, and how do you defend?
• RAGAS
• DeepEval