Policy assistant with citations
Retrieval that refuses when it has no source, and never shows a document to someone who may not read it.
The brief
An HR team fields the same policy questions all week. The policies are spread over dozens of documents, and some, such as salary bands, only managers may see.
What you will build
- Ingestion of a folder of policy documents
- Answers that quote and cite the passage used
- A refusal when nothing relevant is found
- Filtering so each user only retrieves what they may see
Requirements
| Requirement | Done when |
|---|---|
| Citations | Every answer links to the passage it used |
| Refusal | Out-of-scope questions get a clear refusal |
| Permissions | A non-manager never retrieves a manager-only document |
| Freshness | Updating a document updates the answers |
What to learn it from
- Docling: turning documents into clean text, or Unstructured
- LlamaIndex: retrieval with citations, permission filters, hybrid search and reranking
- LangGraph: retrieval-augmented answers
- Topic: SSO, OAuth and RBAC
Milestones
- Collect 20 policy documents and mark two as manager-only
- Index them and answer one question with a citation
- Add the refusal path
- Add the permission filter
- Write 25 test questions, including ones that must be refused
- Measure and fix
How it is judged
A security reviewer asks a non-manager account about salaries and gets nothing. Then you walk them through every hop the data takes.
Stretch goals
- Hybrid keyword and vector search
- Answers that compare two policies
- An admin view of the most asked questions
Before you call it done
Worth remembering
- No answer without a source
- No leak across permissions
- Refusals are helpful, not blank
Every expert started right here.