Batching, flush and shutdown

The SDK queues observations and sends them in batches from a background thread. flush sends the queue now; a short script that skips it can exit with traces unsent.

Every lesson so far called langfuse.flush() before reading the server's list. The reason is part of what the SDK is designed for: tracing must never slow down the application it traces.

import time

answer("Where is my order A17?")
answer("Is B22 on its way?")
answer("Hello?")

print("right away:", len(local_langfuse.SPANS), "observations")
time.sleep(6)
print("6 seconds later:", len(local_langfuse.SPANS), "observations")
print(local_langfuse.REQUESTS)

python batching.py

Right after three tickets, the server had received nothing: the observations were in a queue in your process. About five seconds later the background exporter sent all eight in one request. Five seconds is the default wait, and 512 observations the default batch size; flush_interval and flush_at on the client, or LANGFUSE_FLUSH_INTERVAL and LANGFUSE_FLUSH_AT, change them.

flush and shutdown

flush() sends everything queued and waits until it has been sent. shutdown() flushes and then stops the SDK's background threads. The SDK registers shutdown to run when Python exits normally, so a script that ends without calling either usually still sends its traces.

Usually is the problem. A serverless function that is frozen after it returns, a worker killed by its supervisor, or a process that exits with os._exit never runs that exit hook, and whatever was queued is lost. The documentation's rule for short-lived processes is to call flush() yourself before the end, and shutdown() when a long-running service is told to stop.