Observability: watching every call
Gateway observability is the record the gateway keeps of every call, with its model, cost and timing, and the callbacks that forward that record to the tools you already use.
Last updated: 12 Oct, 2026 · LiteLLM 1.105
Guardrails on every request acted on requests. This lesson reads what the gateway knows about them afterwards. Callbacks: tracking every call showed a callback in the SDK, and the gateway runs the same kind of class for every application behind it, so one callback covers calls from programs you did not write. There are three places to look, from the cheapest to the most complete, and the first two can be run here.
Reading the headers on every response
The gateway adds its own headers to each response. They name the model group that served the request, the model behind it, how many fallbacks were tried, the cost of the call and the gateway version. The spend tracking page documents x-litellm-response-cost as the header that carries the calculated cost. An application can log these headers without calling anything else.
Running your own callback in the gateway
The logging page describes a callback written in a Python file. The class goes in a file, an instance of it is created in the same file, and callbacks in the config names the file and the instance, python_filename.logger_instance_name. The page warns that the dotted path must name the instance, not the class.
from litellm.integrations.custom_logger import CustomLogger
class DeskLogger(CustomLogger):
async def async_log_success_event(self, kwargs, response_obj, start_time, end_time):
print("LOGGED", kwargs["model"], response_obj.usage.total_tokens, flush=True)
logger_instance = DeskLogger()litellm_settings:
callbacks: desk_logger.logger_instanceReading headers and the callback in one run
The example starts a gateway with that callback, makes one call through the OpenAI client with with_raw_response, which gives access to the response headers, prints five gateway headers, and after the gateway stops reads the line the callback printed into gateway.log.
import subprocess
import time
import urllib.request
from pathlib import Path
from openai import OpenAI
def start(port=4400, env=None):
proc = subprocess.Popen(
["litellm", "--config", "config.yaml", "--port", str(port)],
stdout=open("gateway.log", "w"), stderr=subprocess.STDOUT, env=env,
)
while True:
try:
urllib.request.urlopen(f"http://127.0.0.1:{port}/health/liveliness")
return proc
except OSError:
time.sleep(1)
Path("desk_logger.py").write_text(r"""from litellm.integrations.custom_logger import CustomLogger
class DeskLogger(CustomLogger):
async def async_log_success_event(self, kwargs, response_obj, start_time, end_time):
print("LOGGED", kwargs["model"], response_obj.usage.total_tokens, flush=True)
logger_instance = DeskLogger()
""")
Path("config.yaml").write_text("""model_list:
- model_name: careful
litellm_params:
model: groq/openai/gpt-oss-120b
api_key: os.environ/GROQ_API_KEY
litellm_settings:
callbacks: desk_logger.logger_instance
general_settings:
master_key: sk-gateway-1234
""")
proc = start()
try:
client = OpenAI(base_url="http://127.0.0.1:4400", api_key="sk-gateway-1234")
raw = client.chat.completions.with_raw_response.create(
model="careful", max_tokens=200,
messages=[{"role": "user", "content": "Reply with one word: ok"}],
)
for name in ["x-litellm-model-group", "x-litellm-model-name", "x-litellm-attempted-fallbacks",
"x-litellm-response-cost", "x-litellm-version"]:
print(f"{name}: {raw.headers[name]}")
time.sleep(1)
finally:
proc.terminate()
proc.wait()
print([l.strip() for l in open("gateway.log") if l.startswith("LOGGED")])x-litellm-model-group: careful x-litellm-model-name: groq/openai/gpt-oss-120b x-litellm-attempted-fallbacks: 0 x-litellm-response-cost: 3.4949999999999996e-05 x-litellm-version: 1.105.0 ['LOGGED openai/gpt-oss-120b 116']
- The model group is the name the application asked for.
x-litellm-model-groupshowscareful, andx-litellm-model-nameshows the provider model behind it. - No fallback was needed.
x-litellm-attempted-fallbacksis 0 for a request answered by the group it asked for. - The cost came from the gateway. The application did not compute it, and it varies with the tokens of each reply.
- The callback saw the call. The
LOGGEDline is printed by the gateway process, not by the client. The application did not change.
Sending the record to a tool
Most teams already have a tool for this. Langfuse is one: a service that stores and displays traces of LLM calls. OpenTelemetry is a standard format for sending traces to a collector. The logging page shows integrations named in the config. success_callback: ["langfuse"] under litellm_settings logs every successful call to Langfuse, and callbacks: ["otel"] is the form used for OpenTelemetry. For Langfuse the page says to set LANGFUSE_PUBLIC_KEY and LANGFUSE_SECRET_KEY in the environment, with LANGFUSE_HOST optional and defaulting to Langfuse Cloud.
litellm_settings:
success_callback: ["langfuse"]
callbacks: ["otel"]Not run here. These need a Langfuse account or an OpenTelemetry collector, and the block shows no output.
The gateway's own spend log
With a database connected, the gateway also keeps a row per request. The logs page says the log holds spend, token use, key and team for each request, and that request and response content is not stored unless store_prompts_in_spend_logs is turned on. The spend tracking page lists /spend/logs and /spend/keys for reading the rows as JSON. Not run here: this needs the database.
Headers vs callbacks vs the spend log
| Response headers | A callback | The spend log | |
|---|---|---|---|
| Needs | Nothing extra | A file or an integration account | A Postgres database |
| Covers | The call being made | Every call through the gateway | Every call, kept as rows |
| Read by | The application | Your code or the tool you forward to | The dashboard or the spend routes |
Where this fits
- Explaining a bill. The cost header and the spend log say what a call cost, computed in one place.
- One trace for every application. A callback named in the config logs calls from programs that never import a tracing library.
- Spotting fallbacks. A rise in
x-litellm-attempted-fallbackssays a primary model is failing before users complain.
Related
- Previous: Guardrails on every request
- Next: MCP tools through the gateway
- Reference: Logging and Spend tracking in the LiteLLM docs
- Print every header that starts with
x-litellm-fromraw.headers. - Change the callback to print
response_obj.choices[0].message.contentas well, restart the gateway and run again. - Add the
cacheandcache_paramslines from Fallbacks and caching in the gateway config to this config underlitellm_settings, call the gateway twice with the same message, and compare the headers of the two responses.
Every expert started right here.