bind_tools: asking for a tool

A model that knows about a tool can answer with a request to run it instead of text. bind_tools tells the model which tools exist, and the request arrives as tool_calls.

Lesson 3's model could only say it had no way to look an order up. It needs three changes: somewhere to keep the tools it is given, a reply that asks for one, and a way to read the tool's answer when it comes back.

Keeping the tools

Exampleshop_model.py

import re

from langchain.chat_models import BaseChatModel
from langchain.messages import AIMessage, ToolMessage
from langchain_core.outputs import ChatGeneration, ChatResult


class ShopModel(BaseChatModel):
    tools: list = []

    @property
    def _llm_type(self):
        return "shop"

    def bind_tools(self, tools, **kwargs):
        return self.model_copy(update={"tools": tools})

    def _generate(self, messages, stop=None, run_manager=None, **kwargs):
        message = self.decide(messages)
        return ChatResult(generations=[ChatGeneration(message=message)])

bind_tools returns a copy of the model that holds the tool list, the same method every hosted chat model has. _generate now hands the decision to a method called decide, so the rules sit in one place.

Asking for a tool

Exampleshop_model.py, the end of decide

        text = messages[-1].text
        orders = re.findall(r"\b[A-Z]\d+\b", text)
        tool = "refund_order" if "refund" in text.lower() else "lookup_order"
        if orders and tool in [t.name for t in self.tools]:
            calls = [{"name": tool, "args": {"order_id": o}, "id": f"call_{o}"}
                     for o in orders]
            return AIMessage("", tool_calls=calls)
        if orders:
            return AIMessage(f"I have no way to look up {orders[0]} yet.")
        return AIMessage("Hello. Which order is this about?")

When the message names an order and a matching tool was bound, the reply has no text and one tool call per order: the tool's name, its arguments, and an id. A message about a refund asks for refund_order, which arrives in lesson 23.

Reading the result

Exampleshop_model.py, the start of decide

    def decide(self, messages):
        results = []
        for m in reversed(messages):
            if not isinstance(m, ToolMessage):
                break
            results.insert(0, m.text)
        if results:
            return AIMessage(" ".join(results))

A tool's answer comes back as a ToolMessage. When the conversation ends with tool results, the model answers with their text instead of asking again. Put this at the start of decide and save the file; every lesson from here on imports it.

One round trip by hand

Example

from langchain.messages import HumanMessage
from shop_model import ShopModel
from tools import lookup_order

model = ShopModel().bind_tools([lookup_order])
question = HumanMessage("Where is my order A17?")

request = model.invoke([question])
print(request.tool_calls)

No text, one tool call. The model has asked for lookup_order with order_id set to A17, and it is up to your code to run it.

Example

call = request.tool_calls[0]
result = lookup_order.invoke(call)
print(type(result).__name__, result.text)

answer = model.invoke([question, request, result])
print(answer.text)

Invoking a tool with the whole tool call, not only its arguments, returns a ToolMessage tagged with the call's id. Sent back with the question and the request, it gives the model what it needs to answer.

Try it yourself

Bind no tools and invoke the same question. Which reply do you get?
Ask "Where are A17 and C40?" and count the tool calls.
Print result.tool_call_id and compare it with call["id"].

PreviousTools: a function the model can call

Little by little, you're building something great.