RAG Interview Questions
Dashboard
0%
1
Curious builder0 XP earned · 300 to level 2
0 daysFinish a lesson to begin
Badge collection0 of 6 unlocked
51 small wins to finish your pathNext question →

Q36HardSystem design

Design an enterprise knowledge assistant over Confluence, Google Drive, SharePoint and Slack for 10,000 employees.

30-second answerSay your answer out loud first, then reveal.

1. Clarify requirements

  • Sources, volume (say 5M documents, 50M chunks), update frequency.
  • Permissions: must mirror source-system ACLs exactly.
  • Latency SLA (first token < 2s), languages, regions and data residency.
  • Answer types: Q&A with citations, summaries, "who knows about X?"

2. Architecture

Ingestion: Confluence, Drive, SharePoint and Slack connectors sync documents and ACLs into an ingest queue, a parse-chunk-embed pipeline and a hybrid index, plus identity sync; the query path runs from the employee through condense, permission-filtered hybrid retrieval, rerank, grounded LLM answer and post-check, with tracing and evals over both.

3. Key decisions

  • Permission-aware retrieval: store ACL principals (users and groups) on each chunk; resolve the user's groups at query time; filter inside the search engine (Q37). Never rely on post-hoc LLM filtering.
  • Incremental sync: webhooks or change APIs per source, with deletes propagated quickly (a deleted doc must disappear).
  • Source-specific parsing: Slack threads grouped as conversations with channel and time metadata; Confluence pages split by headings; Drive docs and slides with layout parsing.
  • Hybrid + rerank: enterprise queries are full of project codenames and acronyms, so BM25 matters.
  • Authority and recency boosts: official pages over random Slack chatter.
  • Generation: cite with links back to the source; abstain when nothing relevant is found.

4. Scale: 50M chunks × 1024 dims means quantization or sharding; embedding throughput for the initial backfill (batch jobs); per-source rate limits on connectors.

5. Quality loop: golden set per department; thumbs and feedback; logging of "no answer" queries to find content gaps; weekly error analysis.

6. Security and compliance: SSO, audit logs of queries and returned sources, PII redaction policies, data residency, retention aligned with source deletion.

PreviousWhat are Corrective RAG, Self-RAG, and Adaptive RAG?

Little by little, you're building something great.