Query engines: what the model is actually sent

A query engine retrieves chunks, writes them into a prompt with the question, and sends it to a model. A mock model shows that prompt word for word.

from llama_index.core import Settings
from llama_index.embeddings.huggingface import HuggingFaceEmbedding

Settings.embed_model = HuggingFaceEmbedding(model_name="sentence-transformers/all-MiniLM-L6-v2")

import os

from llama_index.core import SimpleDirectoryReader


def only_name(path):
    return {"file_name": os.path.basename(path)}


documents = SimpleDirectoryReader("help", file_metadata=only_name).load_data()

from llama_index.core.node_parser import SentenceSplitter

nodes = SentenceSplitter(chunk_size=80, chunk_overlap=0).get_nodes_from_documents(documents)

from llama_index.core import VectorStoreIndex

index = VectorStoreIndex(nodes)
from llama_index.core.llms import MockLLM

Settings.llm = MockLLM()
query_engine = index.as_query_engine(similarity_top_k=1)
print(query_engine.query("How long does standard delivery take?"))

MockLLM is a stand-in included in LlamaIndex that returns the prompt it was given as its answer, so the printed text is exactly what a real model would read.

Read it. The retrieved chunk is between the dashed lines. No metadata line appears above it: the reader excludes file_name from the model's text by default, as lesson 6 explained, and the citations in the next lesson come from the response instead. Then comes an instruction to answer from the context and not from prior knowledge, then the question. This template is LlamaIndex's default; with a real model it is the whole of what the model knows about your documents.

The three stages

The querying guide describes a query in three stages: retrieval finds the chunks, postprocessing can filter or rerank them, and response synthesis builds the prompt and calls the model. as_query_engine sets up all three with defaults; the next lessons change each one.