Keeping the conversation short

A saved conversation only grows. Left alone it eventually gets too long to send, and you pay for every message on every turn.

Remember from lesson 15 that the whole list goes to the model each time. Turn fifty sends turns one to forty nine along with it. So at some point something has to go.

Keep the last few

The simplest answer is to send only the end of the conversation. trim_messages does that, and it does not touch what is saved.

from langchain_core.messages import HumanMessage, AIMessage, trim_messages

chat = [
    HumanMessage("I was charged twice"),
    AIMessage("Let me look."),
    HumanMessage("Any news?"),
    AIMessage("Refund is on the way."),
]

kept = trim_messages(chat, max_tokens=2, token_counter=len, strategy="last")
print([m.content for m in kept])

token_counter=len counts messages, which keeps this example honest and easy to read. In real code you pass the model itself and it counts tokens, which is what actually costs money.

Nothing was deleted. chat still has four messages in it. You trimmed on the way to the model, which is usually what you want, because the full history is still there if you need it.

Actually forget

Sometimes you do want it gone. Return a RemoveMessage and add_messages takes that message out of the state instead of adding one.

from langchain_core.messages import HumanMessage, AIMessage, RemoveMessage
from langgraph.graph import StateGraph, START, MessagesState

def forget_old(state):
    old = state["messages"][:-2]
    return {"messages": [RemoveMessage(id=m.id) for m in old]}

builder = StateGraph(MessagesState)
builder.add_node("forget_old", forget_old)
builder.add_edge(START, "forget_old")

chat = [HumanMessage("one"), AIMessage("two"), HumanMessage("three"), AIMessage("four")]
result = builder.compile().invoke({"messages": chat})
print([m.content for m in result["messages"]])

This is why add_messages is not simply operator.add, as lesson 13 hinted. It looks at the id on each message, and a RemoveMessage carrying an existing id means take that one out.

Summarise instead of forgetting

Deleting old turns loses whatever was in them. The middle option is to ask the model to write a summary, then keep the summary and drop the messages it covers.

def summarise(state):
    old = state["messages"][:-2]
    summary = model.invoke(old + [HumanMessage("Sum up the conversation so far.")])
    return {"messages": [RemoveMessage(id=m.id) for m in old] + [summary]}

There is no output on that one, because our stand-in model does not write summaries. The shape is what matters: remove the old messages and add one that stands in for them, in a single return.