Retrievers: the search step on its own

Most RAG problems are retrieval problems: the right chunk never reached the model. The retriever on its own shows exactly what came back.

Example

from llama_index.core import Settings
from llama_index.embeddings.huggingface import HuggingFaceEmbedding

Settings.embed_model = HuggingFaceEmbedding(model_name="sentence-transformers/all-MiniLM-L6-v2")

import os

from llama_index.core import SimpleDirectoryReader


def only_name(path):
    return {"file_name": os.path.basename(path)}


documents = SimpleDirectoryReader("help", file_metadata=only_name).load_data()

from llama_index.core.node_parser import SentenceSplitter

nodes = SentenceSplitter(chunk_size=80, chunk_overlap=0).get_nodes_from_documents(documents)

from llama_index.core import VectorStoreIndex

index = VectorStoreIndex(nodes)

retriever = index.as_retriever(similarity_top_k=3)
results = retriever.retrieve("Do used bulbs get refunded?")
for result in results:
    print(f"{result.score:.3f}", result.metadata["file_name"], "|", result.get_content().replace("\n", " ")[:60])

retrieve returns a list of NodeWithScore: the node, and its similarity to the question. The answer is in lamps.md, the sentence about used bulbs, and it came first.

Notice the refund document close behind: the question says "refunded". Retrieval ranks by similarity of meaning, and a chunk about refunds in general is similar to a question about refunds for bulbs. Whether the model gets the right answer depends on the right chunk being first, or at least included.

Look before you blame the model

When an assistant answers wrongly, print what was retrieved for that question first. If the right chunk is missing, no prompt or model will fix it; the fix is in chunking, metadata or the retriever, which parts 2 and 4 cover.

Try it yourself

Ask "When will my express order arrive?" and read the top chunk.
Set similarity_top_k=1 and ask about used bulbs again.
Rebuild the nodes with chunk_size=400 and compare the scores.

PreviousIngestionPipeline: loading, splitting and embedding in one step

Slow is fine. Stopping is the only problem.