Retrieval as a tool

Retrieval as a tool is a pattern where the search sits behind a tool, so the agent decides when to look something up and then answers only from what it found.

Last updated: 27 Sep, 2026 · LangChain 1.4

This is retrieval-augmented generation: find the passages that answer a question, then answer from them. Here the search is a tool, so the model calls it like any other, and when nothing matches it says so instead of guessing.

Building the policy store

Build the store from the previous lesson's chunks and embedding model.

from langchain.tools import tool
from langchain_core.vectorstores import InMemoryVectorStore
from policies import chunks
from word_embeddings import WordEmbeddings

store = InMemoryVectorStore(WordEmbeddings())
store.add_documents(chunks)

The search tool

Put the search behind a tool. It keeps the chunks scoring 0.3 or more and returns them with their sources; below that it returns a fixed sentence, so there is nothing to invent from.

@tool
def search_policies(query: str) -> str:
    """Search the shop's policies on refunds, shipping and accounts."""
    found = [doc for doc, score in store.similarity_search_with_score(query, k=2) if score >= 0.3]
    if not found:
        return "No policy covers this."          # nothing to answer from
    return "\n".join(f"[{doc.metadata['source']}] {doc.page_content}" for doc in found)

A model that quotes the source

A small model turns every question into a search, quotes a result with its source, and turns the fixed sentence into a reply that admits the gap.

from langchain.messages import AIMessage
from shop_model import ShopModel

class HelpModel(ShopModel):
    def decide(self, messages):
        last = messages[-1]
        if last.type == "tool" and last.text == "No policy covers this.":
            return AIMessage("Our policies do not cover that. A person from the team will reply.")
        if last.type == "tool":
            return AIMessage(f"From our policies:\n{last.text}")
        search = {"name": "search_policies", "args": {"query": last.text}, "id": "call_search"}
        return AIMessage("", tool_calls=[search])   # every question goes to the search

Giving the agent the tool

Give the model the tool and let the agent loop run.

from langchain.agents import create_agent
from help_model import HelpModel
from search import search_policies

agent = create_agent(HelpModel(), tools=[search_policies])

Running a covered and an uncovered question

Ask one question the policies cover and one they do not.

for question in ["How long does a refund take?", "Can I pay with bitcoin?"]:
    result = agent.invoke({"messages": [{"role": "user", "content": question}]})
    print(result["messages"][-1].text, end="\n\n")

The refund question found the refund policy and the answer names its file. The bitcoin question scored under the cut, so the reply says the policies do not cover it. That second answer is the one that keeps a support desk trustworthy.

How the score decides the answer

• The refund question scored above the cut, so the tool returned the refund chunk and the answer names its file.
• The bitcoin question scored under 0.3, so the tool returned its fixed sentence and the reply admits the gap instead of guessing.
• The model answers only from what the tool returned, which is what keeps the answer grounded and checkable.

Plain model vs retrieval tool

When to use retrieval as a tool

• A support desk that must answer from fixed policies and refuse the rest.
• Any answer that has to be grounded in your data, with a source, not guessed.

Related

• Previous: Embeddings and a vector store
• Next: MCPAdapter: tools from another program
• Reference: Retrieval