A model you can run for free

Every lesson from here needs a model. Rather than ask you for a credit card, you are going to write one.

A chat model, from your program's point of view, is an object with an invoke method. Messages go in, one AIMessage comes out. What happens in between is somebody else's problem, which means you can write a stand-in. Call it PretendModel.

It will not be intelligent. It will be a real chat model in every way your graph can tell, and it will genuinely decide, which is the part that matters for learning. Build it in five short pieces.

One: what it is made of

from langchain_core.language_models import BaseChatModel
from langchain_core.messages import AIMessage
from langchain_core.outputs import ChatGeneration, ChatResult

BaseChatModel is the class every chat model in LangChain inherits from, and inheriting from it is what makes this object a real model to everything else. The other two are wrappers a model has to put its message inside, and you will see them used once.

Two: a small helper

The model needs to spot an order id in a sentence. This is not clever, and it does not need to be.

def _first_id(text):
    """The first word with a digit in it, which is close enough to an id."""
    for word in text.replace("?", " ").replace(".", " ").replace(",", " ").split():
        if any(c.isdigit() for c in word):
            return word
    return ""

Three: the shell

Two of these three things are required of any chat model. The third is the one that matters for lesson 17.

class PretendModel(BaseChatModel):
    tools: list = []

    @property
    def _llm_type(self):
        return "pretend"

    def bind_tools(self, tools, **kwargs):
        return PretendModel(tools=list(tools))

tools holds whatever has been attached, and it stays empty until something is bound. _llm_type is a required name that nothing really looks at. bind_tools is the one that matters: it attaches tools and hands back a new model that knows about them, and lesson 17 is entirely about it.

Four: the one required method

Every chat model has to provide _generate. It takes the conversation and returns a result wrapping a single message.

    def _generate(self, messages, stop=None, run_manager=None, **kwargs):
        return ChatResult(generations=[ChatGeneration(message=self._decide(messages))])

Four rules, read top to bottom. If a tool has just answered, say what it found. If the message opens with a passage of text, answer out of that passage, which is what lesson 32 uses to answer questions about a set of documents. Otherwise, if a word from a tool's name appears in the question, ask for that tool, though this can never fire until something is bound. If nothing matches, say which id was spotted and ask for what is missing.

Five: the fake part

Four rules, top to bottom. Read them once and you will always know why this model answered the way it did.

    def _decide(self, messages):
        last = messages[-1]
        if last.type == "tool":
            return AIMessage(f"Here is what I found. {last.content}")
        text = str(last.content)
        if text.startswith("Context:"):
            passage = text.split("Context:", 1)[1].split("Question:")[0].strip()
            return AIMessage(f"According to the documentation: {passage}")
        for t in self.tools:
            if any(w in text.lower() for w in t.name.split("_")):
                args = {}
                for name, spec in t.args.items():
                    value = _first_id(text)
                    args[name] = int(value) if spec.get("type") == "integer" else value
                return AIMessage("", tool_calls=[{"name": t.name, "args": args, "id": "call_1"}])
        found = _first_id(text)
        if found:
            return AIMessage(f"I can see the id {found}, but I have no way to look it up.")
        return AIMessage("I can help with that. What is the order id?")

Three rules, read top to bottom. If a tool has just answered, say what it found. Otherwise, if a word from a tool's name appears in the question, ask for that tool, though this can never fire until something is bound. If neither matches, say which id was spotted and ask for what is missing.

Save all five pieces as pretend_model.py, next to the files you are writing. Every example after this one imports it, and you can also download it here.

Watch it decide

from pretend_model import PretendModel
from langchain_core.messages import HumanMessage

model = PretendModel()

for question in ["Where is order A17?", "Hello there"]:
    print(repr(model.invoke([HumanMessage(question)]).content))

Two questions, two different answers, and the model chose between them by reading the text. It spotted A17 in the first and said so. It has no tools yet, so it cannot do anything with that id, which is the gap lesson 17 fills.