RAG Interview Questions
Dashboard
0%
1
Curious builder0 XP earned · 300 to level 2
0 daysFinish a lesson to begin
Badge collection0 of 6 unlocked
51 small wins to finish your pathNext question →

Q44HardSystem design

Design a documentation assistant that answers coding questions using the latest library docs and source code (version-aware).

30-second answerSay your answer out loud first, then reveal.
A version-aware documentation assistant: GitHub repos and docs sites feed a release watcher, parsing and metadata tagging into a hybrid index with a symbol table, which a query pipeline searches with library and version filters before reranking, generating cited code and optionally checking it in a sandbox.

Key decisions

  1. Version awareness is the core problem. LLMs often produce deprecated APIs from their training data. Tag every chunk with library and version; filter by the requested version; include changelog and migration-guide chunks ("X was renamed to Y in 2.0").
  2. Exact identifier matching: function and class names ("create_react_agent") need BM25 or a symbol-table lookup, not only semantics.
  3. Code-aware chunking: chunk by symbol with its signature, docstring and examples; keep code blocks intact.
  4. Source of truth hierarchy: official API reference > official guides > examples > community content.
  5. Freshness: trigger re-indexing on new releases (GitHub release webhooks or tag polling); keep older versions for users on old versions.
  6. Grounded code generation: instruct the model to use only APIs present in the retrieved docs; optionally verify by static analysis (does the symbol exist in that version's source?) or by executing the snippet in a sandbox.
  7. Delivery: IDE integration or MCP server so coding agents can query it as a tool.

Eval: questions about recently changed APIs (where base LLMs fail), measured by whether the generated code runs and uses non-deprecated APIs.

PreviousWith 1M+ token context windows, is RAG dead? Argue both sides.

This is what real progress feels like.