FDE take-home: "Build a prototype that answers questions over these 50 documents." How do you structure your submission?

Recommended structure

README.md            ← start here: what, how to run, results, decisions, next steps
app/                 ← clean, modular code (ingest, retrieve, answer, api/ui)
evals/
  questions.jsonl    ← 20-40 Q&A pairs incl. unanswerable + tricky ones
  run_evals.py       ← one command; outputs a metrics table
  results.md         ← scores + error analysis by category
tests/               ← a few unit tests (parsing, retrieval, citation validation)
docker-compose.yml   ← or make run / uv run; one-command start

README sections

1. Problem understanding and assumptions ("Assumed users are support agents; English only").
2. Architecture diagram + key choices (why hybrid search, chunk size, model choice).
3. Results: retrieval recall@k, answer correctness, faithfulness, latency, cost per query.
4. Error analysis: what fails and why.
5. Limitations and risks (honest).
6. Production next steps: auth and permissions, monitoring, scaling, security, eval expansion, feedback loop.

Signals that stand out

• An eval harness (most candidates skip it).
• Citations plus "I don't know" behaviour.
• Thoughtful trade-offs explained concisely.
• Clean, runnable code within the time guideline. Don't spend 3x the suggested time; mention what you'd do with more.

Related

• Previous: Q47. FDE live-coding round: "Build a minimal tool-calling agent in 45 minutes." What do you build, and what do interviewers look for?
• Next: Q49. Behavioural: "Tell me about a time you pushed back on a customer." How should you structure your answer, and what does a strong answer sound like?
• RAG Interview Questions
• RAGAS
• LlamaIndex