Runtime context: who is asking

Some facts a tool needs should never come from the model, such as who the customer is. Runtime context passes them in with each call, out of the model's sight.

Lesson 7's agent tells anyone the status of any order. A customer called Ravi should see his own orders and no one else's. The name cannot be a tool argument, because the model fills in arguments and a message can talk a model into sending any name.

A schema for the context

from dataclasses import dataclass


@dataclass
class Customer:
    name: str

The context is whatever your code knows about this run: the signed-in customer, a database connection, a setting. A dataclass describes its shape.

A tool that reads it

from langchain.tools import ToolRuntime, tool

ORDERS = {"A17": ("ravi", "shipped on 3 March"), "C40": ("mei", "waiting for stock")}


@tool
def lookup_order(order_id: str, runtime: ToolRuntime[Customer]) -> str:
    """Look up one of the customer's orders by its id, such as A17."""
    owner, status = ORDERS.get(order_id, (None, None))
    if owner != runtime.context.name:
        return f"{order_id} is not one of your orders."
    return f"{order_id} {status}."

A parameter typed ToolRuntime is filled in by LangChain when the tool runs. runtime.context is the object you passed for this run, so the tool compares the order's owner with the customer asking.

from tools import lookup_order

print(lookup_order.args)

The schema the model sees has only order_id. The runtime parameter is left out of it, so the model cannot supply or change the customer.

Passing the context in

from langchain.agents import create_agent
from shop_model import ShopModel
from tools import Customer, lookup_order

agent = create_agent(ShopModel(), tools=[lookup_order], context_schema=Customer)

result = agent.invoke({"messages": [{"role": "user", "content": "Where is my order A17?"}]}, context=Customer("ravi"))
print(result["messages"][-1].text)

result = agent.invoke({"messages": [{"role": "user", "content": "Where is my order A17?"}]}, context=Customer("mei"))
print(result["messages"][-1].text)

context_schema=Customer tells the agent what to expect, and context= supplies it for one call. The same question gets two answers: A17 is Ravi's order, so Mei is told it is not hers.