LLM calls: prompts, completions and tokens

init patches installed LLM libraries. An OpenAI call becomes a span with the model, the prompt, the completion and token counts, without any decorator.

from openai import OpenAI

client = OpenAI(base_url=f"{url}/v1", api_key="not-a-real-key")


@agentops.trace(name="support-ticket")
def handle(ticket):
    reply = client.chat.completions.create(model="gpt-4.1-mini", messages=[{"role": "user", "content": ticket}])
    return reply.choices[0].message.content

print(handle("I was charged twice for order A-1001"))
local_collector.flush()
llm = next(span for span in local_collector.SPANS if span["name"] == "openai.chat.completion")
for key, value in llm["attributes"].items():
    if key.startswith("gen_ai."):
        print(key, "=", value)

python llm.py

The OpenAI client is pointed at the collector's fake API with base_url, so no request goes to OpenAI and the key is a placeholder. AgentOps recorded the call as openai.chat.completion inside the trace: gen_ai.system, the requested and returned model, the prompt messages as gen_ai.prompt.N, the answer as gen_ai.completion.N, and token usage from the response. These names follow OpenTelemetry's gen_ai conventions.

Leaving prompt text out

python llm_no_content.py

With TRACELOOP_TRACE_CONTENT=false set before AgentOps is imported, the prompt and completion text are not recorded, while the model and token counts still are. The variable's name comes from OpenLLMetry, the instrumentation AgentOps builds on. It only covers LLM calls: decorator inputs, lesson 4, are still recorded.