Project: chat with your documents
A document chat app is a RAG pipeline with two entry points: an indexing function that turns files into a saved vector index once, and a chat function that condenses each question with the history, retrieves, answers from the chunks only and returns the sources.
Last updated: 08 Oct, 2026 · LangChain 1.4
This project joins the parts of the section: Loading and chunking documents, Embeddings and vector stores, Building a RAG pipeline and Conversational RAG. It runs on three public arXiv papers that the code downloads, keeps only a few pages of each so that indexing stays small, and needs two keys: GROQ_API_KEY for the chat model and GOOGLE_API_KEY for the embeddings.
pip install langchain langchain-groq langchain-google-genai langchain-community langchain-text-splitters faiss-cpu pypdfPlanning the files
The video makes a src folder with one file per job: a data loader that reads the documents, a vector store file, an embedding file and a search file, with an app.py that wires them. The same four jobs appear below as pieces of one script, so that the whole app runs as a single example. Splitting them into files later is a matter of moving functions.
| Video's file | Job | In this project |
|---|---|---|
| data_loader | Read every document | The loop over PAPERS in build_index |
| embedding | Split and embed the chunks | The splitter and GoogleGenerativeAIEmbeddings |
| vectorstore | Build, save and load the index | FAISS.from_documents, save_local, load_local |
| search | Retrieve and answer | The chat function |
Building the index once
The video builds the FAISS index the first time and saves it in a folder, then loads it on later runs, so that the chunks are not embedded again. The video's store holds hand-made vectors from a local model with an L2 index; here LangChain's FAISS does the same work with Gemini embeddings. build_index returns the loaded store when the folder exists.
Loading and splitting the pages
pages = []
for name, arxiv_id in PAPERS.items():
if not os.path.exists(f"{name}.pdf"):
urllib.request.urlretrieve(f"https://arxiv.org/pdf/{arxiv_id}", f"{name}.pdf")
docs = PyPDFLoader(f"{name}.pdf").load()
pages += [docs[i] for i in PICK[name]]
chunks = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=200).split_documents(pages)Embedding, saving and reloading
store = FAISS.from_documents(chunks, embeddings, ids=[f"c{i}" for i in range(len(chunks))])
store.save_local(INDEX_DIR) # next run: FAISS.load_local(INDEX_DIR, embeddings, ...) insteadChatting against the saved index
The video ends by adding a search step that queries the store and sends the context to a Groq model, and it calls the result the whole crash course in one pipeline. The chat function below does that with the history step from the conversational lesson. It numbers pages from 1 for people, tells the model to say when the documents hold no answer, and returns the sources next to the reply.
The chat function
def chat(question):
standalone = (condense | model).invoke({"history": history, "question": question}).content if history else question
docs = retriever.invoke(standalone)
context = "\n\n".join(f"[{d.metadata['source']} p.{d.metadata['page'] + 1}] {d.page_content}" for d in docs)
reply = (answer | model).invoke({"context": context, "history": history, "question": question}).content
history.extend([HumanMessage(question), AIMessage(reply)])
sources = sorted({(d.metadata["source"], d.metadata["page"] + 1) for d in docs})
return reply, sourcesThe app end to end
The conversation below is scripted so that it can run without typing. It asks about the Transformer paper, follows up with a pronoun, switches to the object detection review, and ends with a question no document answers.
import os, urllib.request
from langchain_community.document_loaders import PyPDFLoader
from langchain_text_splitters import RecursiveCharacterTextSplitter
from langchain_community.vectorstores import FAISS
from langchain_google_genai import GoogleGenerativeAIEmbeddings
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.messages import HumanMessage, AIMessage
from langchain.chat_models import init_chat_model
PAPERS = {"attention": "1706.03762", "qzhou": "2508.21632", "objdet": "1807.05511"}
PICK = {"attention": [3, 4], "qzhou": [0, 1], "objdet": [0]}
INDEX_DIR = "paper_index"
embeddings = GoogleGenerativeAIEmbeddings(model="gemini-embedding-001")
model = init_chat_model("groq:openai/gpt-oss-120b", temperature=0)
def build_index():
if os.path.exists(INDEX_DIR):
print("loading the saved index")
return FAISS.load_local(INDEX_DIR, embeddings, allow_dangerous_deserialization=True)
pages = []
for name, arxiv_id in PAPERS.items():
if not os.path.exists(f"{name}.pdf"):
urllib.request.urlretrieve(f"https://arxiv.org/pdf/{arxiv_id}", f"{name}.pdf")
docs = PyPDFLoader(f"{name}.pdf").load()
pages += [docs[i] for i in PICK[name]]
chunks = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=200).split_documents(pages)
store = FAISS.from_documents(chunks, embeddings, ids=[f"c{i}" for i in range(len(chunks))])
store.save_local(INDEX_DIR)
print("built an index of", len(chunks), "chunks from", len(pages), "pages")
return store
store = build_index()
retriever = store.as_retriever(search_kwargs={"k": 3})
condense = ChatPromptTemplate.from_messages([
("system", "Rewrite the final question as one standalone question that makes sense without the conversation. Do not answer it. Return only the rewritten question."),
("placeholder", "{history}"), ("human", "{question}")])
answer = ChatPromptTemplate.from_messages([
("system", "Answer in at most three sentences, using only the context. If the answer is not in the context, say you cannot find it in the documents.\n\nContext:\n{context}"),
("placeholder", "{history}"), ("human", "{question}")])
history = []
def chat(question):
standalone = (condense | model).invoke({"history": history, "question": question}).content if history else question
docs = retriever.invoke(standalone)
context = "\n\n".join(f"[{d.metadata['source']} p.{d.metadata['page'] + 1}] {d.page_content}" for d in docs)
reply = (answer | model).invoke({"context": context, "history": history, "question": question}).content
history.extend([HumanMessage(question), AIMessage(reply)])
sources = sorted({(d.metadata["source"], d.metadata["page"] + 1) for d in docs})
return reply, sources
for q in ["What is scaled dot-product attention?", "Why is it divided by that square root?",
"What does the object detection review say about handcrafted features?", "What is the capital of Australia?"]:
reply, sources = chat(q)
print("YOU:", q)
print("AI :", reply)
print("sources:", sources)
print()
print(len(history), "messages in history")built an index of 23 chunks from 5 pages
YOU: What is scaled dot-product attention?
AI : Scaled Dot‑Product Attention takes queries Q, keys K (both of dimension d_k) and values V (dimension d_v), computes the dot products QKᵀ, scales them by 1/√d_k, applies a softmax to obtain attention weights, and then multiplies these weights by V (Attention(Q,K,V)=softmax(QKᵀ/√d_k) V). This scaling prevents the dot products from becoming too large, which would push the softmax into regions with very small gradients. It is a fast, matrix‑multiplication‑friendly form of attention.
sources: [('attention.pdf', 4)]
YOU: Why is it divided by that square root?
AI : Because the dot product of a query and a key has variance d_k (it grows with the dimension), its magnitude can become large and push the softmax into regions with extremely small gradients. Dividing by √d_k normalizes the scale of the dot products, keeping the softmax’s input in a stable range. This counteracts the gradient‑vanishing problem described in the document.
sources: [('attention.pdf', 4)]
YOU: What does the object detection review say about handcrafted features?
AI : The review notes that traditional object‑detection methods rely on handcrafted features combined with shallow, trainable architectures. However, it observes that such approaches quickly reach a performance plateau, often requiring complex ensembles of low‑level features and high‑level context to achieve modest gains. Consequently, handcrafted‑feature‑based detectors are considered limited compared with newer deep‑learning models.
sources: [('objdet.pdf', 1)]
YOU: What is the capital of Australia?
AI : I cannot find that information in the provided documents.
sources: [('attention.pdf', 4), ('qzhou.pdf', 1)]
8 messages in historyReading the conversation
- The first run built the index: 23 chunks from 5 pages, saved to
paper_index. A later run printsloading the saved indexand embeds nothing. - The first answer defines scaled dot-product attention, with the source
attention.pdfpage 4. Pages count from 1 here, becausechatadds 1 to the loader's page index. - The pronoun follow-up "Why is it divided by that square root?" was condensed with the history, retrieved the same page, and the answer explains the key dimension and the softmax.
- The third question moves to another document: the source is
objdet.pdfpage 1, and the answer repeats the abstract's point about handcrafted features and shallow trainable architectures. - The last question has no answer in any document. The reply is
I cannot find that information in the provided documents., yet the sources line still lists two chunks, because retrieval always returns something. A UI should hide the sources when the model declines. - Eight messages are in the history after four turns, two for each turn.
Typing the questions yourself
Swap the scripted list for a loop that reads from the keyboard. It uses the same chat function and stops on an empty line.
while True:
question = input("You: ").strip()
if not question:
break
reply, sources = chat(question)
print("AI:", reply)
print("Sources:", sources)The video's app vs this app
| In the video | In this project | |
|---|---|---|
| Embeddings | A local sentence-transformers model | Hosted gemini-embedding-001 |
| Index | A hand-made FAISS index with L2 distance, saved as a file with a metadata pickle | FAISS through LangChain, saved with save_local |
| Model | A Groq Gemma model, since retired | Groq openai/gpt-oss-120b |
| Questions | One query per run | A conversation with history |
| Documents | PDFs placed in a data folder | Three public papers downloaded by URL |
| Keys | Pasted into the file | Read from the environment |
Where you use a document chat app
- Team knowledge bases. Point
PAPERSat policies or manuals and keep the index folder next to the app. - Study tools. Questions over lecture notes or papers, with the page of each answer.
- A base to grow. Add the hybrid search from the better-retrieval lesson, or move retrieval into a tool as in the agentic lesson.
paper_index folder, or index into a new folder, after changing PAPERS or PICK. allow_dangerous_deserialization=True is needed because FAISS stores its metadata as a pickle, which can run code when loaded, so only load index folders you created yourself.Related
- Previous: Evaluating RAG
- Next: AI agents
- See also: Agentic RAG
- Change
PICKto keep pages 5 and 6 of the Transformer paper, delete thepaper_indexfolder, and ask about the feed-forward network. - Ask a follow-up that depends on the previous answer, then print
historyto see what the condense step was given. - Change
kto 1 and ask the object detection question again, then read the sources line.
Every expert started right here.