Summarizing long conversations

summarize_messages keeps a conversation under a token budget by replacing its oldest messages with a summary, and remembers which messages it already summarized.

from langchain_core.messages import AIMessage, HumanMessage
from langmem.short_term import summarize_messages

from memory_model import MemoryModel

messages = []
for i in range(5):
    messages.append(HumanMessage(f"Where is order A-100{i}? It has been a week, please check.", id=f"q{i}"))
    messages.append(AIMessage(f"Order A-100{i} is on its way and should arrive soon.", id=f"a{i}"))

result = summarize_messages(messages, running_summary=None, model=MemoryModel(), max_tokens=60, max_summary_tokens=30)
for message in result.messages:
    print(f"{message.type:6} {message.content}")
print(sorted(result.running_summary.summarized_message_ids))

Ten messages were over max_tokens=60, counted with LangChain's approximate counter. summarize_messages chose the oldest nine to summarize and returned a system message with the summary plus the recent message that fit. running_summary records the summary and the ids it covered; messages need ids for that.

The summary names only orders A-1003 and A-1004, yet all nine messages are marked as summarized. The source explains it: the messages sent to the model for summarizing are trimmed to fit max_tokens as well, dropping the oldest first. With a budget this small, the first three orders were lost. Real budgets are thousands of tokens, where this only happens to very long backlogs.

messages.append(HumanMessage("Thanks, that helps.", id="q5"))
again = summarize_messages(messages, running_summary=result.running_summary, model=MemoryModel(), max_tokens=60, max_summary_tokens=30)
print(again.running_summary is result.running_summary)
print([message.type for message in again.messages])

On the next turn, passing the running summary back means already summarized messages are not summarized again. Still within budget, it returned the same summary object and added the new message. Store running_summary with the thread, for example in LangGraph state.