Generative AI and Agentic AILangChain 1.4 · LangGraph 1.2 · Python 3.12 or 3.13
Dashboard
0%

Vectorless RAG

Vectorless RAG is a retrieval method without embeddings: an LLM builds a table-of-contents tree of a document once, and at question time an LLM reads that tree to choose which sections to read and answer from.

Last updated: 08 Oct, 2026 · LangChain 1.4

Vector RAG cuts a document into fixed-size chunks and searches their vectors. That can split a section in the middle and returns a similarity score that explains little. The video's second approach, built around the PageIndex library, keeps whole sections and lets a model navigate them the way a person uses a table of contents.

Building: an LLM tree builder reads a long PDF and writes a JSON tree index with a title, page range and summary per section. Answering: an LLM reads the tree for the question and picks sections, those pages are fetched, and an LLM answers with page citations. No embeddings or vector store are used.

How the tree index works

How vectorless RAG works: the tree builder and the tree search · from the Complete Agentic AI Course video · 7:15:42 to 7:20:58

The video starts from a PDF that has a table of contents: chapters with page numbers, subsections under them. An LLM tree builder turns that into a hierarchy of nodes saved as a JSON tree index. Each node stands for a section and holds its title and page range, and a short summary of its content written by the LLM. The PageIndex tree also keeps the section text on the node, so a search prompt can show the model the summaries and the answer step can read the full text.

At question time the model is given the whole tree as context, decides which nodes are likely to hold the answer, and the text of those sections is passed to a model that writes the answer with section and page citations. The video's human analogy fits: open the book at the table of contents, find the page, read it. No database server is involved, and the tree can live in a file, an object store or a document database.

When the PDF has no table of contents

Documents without a table of contents, and the retrieval loop · from the Complete Agentic AI Course video · 7:22:25 to 7:26:05

Many PDFs have no table of contents. The video's answer is that an LLM reads the pages, infers the headings and splits at logical boundaries, not at a token count, so a section is not cut into pieces the way a fixed-size chunker would. The sections still correspond to page ranges, so this is chunking by structure rather than no splitting at all. The video then lists the retrieval loop: read the tree, reason about which nodes to select, extract their content, and check whether it is enough to answer; if it is not, go back to selecting. The code below makes one pass; the sufficiency check can be added as one more model call.

The video's run with the PageIndex library

The video's tree search and cited answer on a syllabus PDF · from the Complete Agentic AI Course video · 7:34:48 to 7:38:45

The video runs the PageIndex cloud service on a long syllabus PDF with an OpenAI model: it uploads the file, waits for the tree, prints it and counts the nodes, then writes a tree-search prompt, selects the nodes for a question and generates a cited answer. Shown as it ran in the video, not run here: it needs a PageIndex key and an OpenAI key. The current library names a node's pages start_index and end_index, where the video's notebook reads page_index.

python
from pageindex import PageIndexClient                # pip install pageindex

client = PageIndexClient(api_key="YOUR_PAGEINDEX_KEY")
doc_id = client.submit_document("report.pdf")["doc_id"]   # upload; the tree is built in the background
# poll client.get_document(doc_id)["status"] until it is "completed"
tree = client.get_tree(doc_id, node_summary=True)["result"]   # nodes with titles, ids, summaries, pages

The same three steps work with only the course keys if the tree is built by the chat model. The example builds a small tree of the Transformer paper, asks the model which sections answer a question, fetches those pages and asks for a cited answer.

Building a tree with the chat model

What a section looks like

python
from pydantic import BaseModel

class Section(BaseModel):
    node_id: str
    title: str
    start_page: int
    end_page: int
    summary: str

class Tree(BaseModel):
    sections: list[Section]

Showing the model each page

A real tree builder reads whole pages. To keep the request small, the model here is shown, for every page, its first 150 characters and any numbered heading lines that a regular expression finds, such as 3.2.1 Scaled Dot-Product Attention. Opening text alone gives the model little sign of where the paper's sections begin; the headings let it start a section where the paper does, and the last try-it removes them so that you can compare.

python
import re

def digest_line(i, text):
    headings = re.findall(r"^\d+(?:\.\d+)* [A-Z][^\n]{3,70}$", text, flags=re.M)
    return f"[page {i + 1}] starts: {text[:150]!r} headings: {headings}"

digest = "\n".join(digest_line(i, p) for i, p in enumerate(pages))

Asking the model for the tree

The model returns a table of contents as the Tree schema. method="json_schema" makes Groq return JSON that fits it.

python
tree = model.with_structured_output(Tree, method="json_schema").invoke(
    "Build a table of contents of 8 to 12 sections. Each section starts on the page where its heading "
    "appears and ends on the page where the next section starts...\n\n" + digest
).model_dump()

Searching the tree and answering with pages

python
pick = model.with_structured_output(Pick, method="json_schema").invoke(
    "Question: " + question + "\nTable of contents:\n" + json.dumps(tree["sections"])
    + "\nWhich sections most likely contain the answer? Return their ids."
)
chosen = [s for s in tree["sections"] if s["node_id"] in pick.node_ids]

A tree of the Transformer paper, from build to cited answer

ExampleAPI keyDownloads the Transformer paper by URL; run on Groq openai/gpt-oss-120b
import os, re, json, urllib.request
from pydantic import BaseModel
from langchain_community.document_loaders import PyPDFLoader
from langchain.chat_models import init_chat_model

if not os.path.exists("attention.pdf"):
    urllib.request.urlretrieve("https://arxiv.org/pdf/1706.03762", "attention.pdf")
pages = [d.page_content for d in PyPDFLoader("attention.pdf").load()]
model = init_chat_model("groq:openai/gpt-oss-120b", temperature=0)

class Section(BaseModel):
    node_id: str
    title: str
    start_page: int
    end_page: int
    summary: str

class Tree(BaseModel):
    sections: list[Section]

class Pick(BaseModel):
    thinking: str
    node_ids: list[str]

def digest_line(i, text):
    headings = re.findall(r"^\d+(?:\.\d+)* [A-Z][^\n]{3,70}$", text, flags=re.M)   # lines like "3.2.1 Scaled Dot-Product Attention"
    return f"[page {i + 1}] starts: {text[:150]!r} headings: {headings}"

digest = "\n".join(digest_line(i, p) for i, p in enumerate(pages))
tree = model.with_structured_output(Tree, method="json_schema").invoke(
    "Below is the opening text and the numbered headings found on each page of a paper. Build a table of "
    "contents of 8 to 12 sections. Each section starts on the page where its heading appears and ends on the "
    "page where the next section starts; use the heading numbers in the titles. Give each a one-sentence summary "
    "and ids 0001, 0002, ...\n\n" + digest
).model_dump()
for s in tree["sections"]:
    print(s["node_id"], f"p.{s['start_page']}-{s['end_page']}", s["title"], "|", s["summary"])

question = "Why does the Transformer divide the dot products by the square root of the key dimension?"
pick = model.with_structured_output(Pick, method="json_schema").invoke(
    "Question: " + question + "\nTable of contents:\n" + json.dumps(tree["sections"])
    + "\nWhich sections most likely contain the answer? Return their ids."
)
print(pick)
chosen = [s for s in tree["sections"] if s["node_id"] in pick.node_ids]
text = "\n".join(f"[page {n}]\n{pages[n - 1]}" for s in chosen for n in range(s["start_page"], s["end_page"] + 1))
print("pages read:", sorted({n for s in chosen for n in range(s["start_page"], s["end_page"] + 1)}), "characters:", len(text))
reply = model.invoke("Answer the question using only the pages below. Cite page numbers like (p.N). Max three sentences."
                     "\n\nQuestion: " + question + "\n\n" + text)
print(reply.content)

Reading the tree, the choice and the answer

  • The tree has nine sections, each with an id, a page range and a one-sentence summary. The titles carry the heading numbers (3.2 Attention) because the digest handed them to the model.
  • The ranges are approximate. They overlap where a section begins in the middle of a page: 0003 covers pages 2 to 3, 0004 page 3, 0005 pages 3 to 6. A tree built from a page-by-page digest is a guide to where to read, not an exact map.
  • The model chose one node, 0005, and its thinking field says why: the scaling factor is explained in the attention description, section 3.2. The choice and the reasoning are visible, which a similarity score does not give.
  • Pages 3 to 6 were read, 10,984 characters, the whole section and not a slice of it.
  • The answer cites page 4 twice. That is the page of the PDF where the paper explains the scaling, so the citation can be checked by opening it.

Choosing between vector and vectorless RAG

When to use traditional RAG, when to use vectorless, and hybrid · from the Complete Agentic AI Course video · 7:59:14 to 8:01:34

The video's guidance, in a table: traditional RAG for very large and mixed collections, tight latency and short factual questions; vectorless for long structured documents such as annual reports and contracts, where explaining the path matters. It ends with the hybrid view: scale from vectors plus structure from the tree.

Vector RAGVectorless RAG
What is indexedFixed-size chunks and their embeddingsA tree of sections with titles, page ranges and summaries
Work at question timeOne embedding and one searchAt least one model call to read the tree, one more to answer
Collection sizeMillions of mixed documentsOne to thousands of long documents
Why a result was chosenA similarity scoreThe model's stated reasoning and a section path
Weak spotA section split across chunks loses contextMore latency and cost per question; needs documents with structure
Typical useSupport articles, tickets, web pagesAnnual reports, legal contracts, long manuals

Where you use vectorless RAG

  • Annual reports and filings. A question about one section of a 200-page document, where the answer needs the whole section.
  • Contracts and policies. The reader wants the clause and its page, and the path to it.
  • Long manuals. The structure is already a table of contents, so the tree is cheap to build.
Watch out. The tree here depends on a regular expression for headings and on what the model makes of them, and the search sends the whole tree on every question. A real library reads more of each page and builds deeper trees, and a very large collection would not fit in one prompt. When there are millions of documents, search by vectors first and use the tree inside the few documents that come back.
Try it yourself
  • Change the question to "How many attention heads does the model use?" and read which sections the model picks.
  • Remove headings from digest_line and compare the section titles and page ranges with the first tree.
  • Ask for 14 to 18 sections instead of 8 to 12 and see how the page ranges narrow.
PreviousAgentic RAG

Slow is fine. Stopping is the only problem.