Tracing OpenAI calls without changing them

Importing OpenAI from langfuse.openai instead of openai records every chat call as a generation, with the messages, model, parameters and token usage.

Lesson 6 wrote the generation by hand. Most applications call a model through a client library, and Langfuse ships drop-in versions of some of them. The OpenAI one is a changed import; the calls stay the same. The model here is still your reply, served by the local server behind an OpenAI-style API.

from langfuse import Langfuse, observe
from langfuse.openai import OpenAI

import local_langfuse
from shop_model import reply

url = local_langfuse.start(model=reply)
langfuse = Langfuse(public_key="pk-lf-local", secret_key="sk-lf-local", base_url=url)
client = OpenAI(base_url=f"{url}/v1", api_key="not-a-real-key")

start(model=reply) makes the local server answer /v1/chat/completions with your function. The client's base_url points there, so the key can be any string; with a real provider it would be that provider's address and key.

import re

ORDERS = {"A17": "shipped on 3 March"}
SYSTEM = "You answer customers of a small online shop in one short sentence."


@observe(name="lookup-order", as_type="tool")
def lookup_order(order_id):
    return ORDERS.get(order_id, "not found")

def ask_model(messages):
    response = client.chat.completions.create(model="shop-model", messages=messages, name="write-reply")
    return response.choices[0].message.content

This is an ordinary OpenAI call. name is the one addition, a keyword the Langfuse wrapper accepts to name the generation; the answer function is lesson 6's, unchanged.

print(answer("Where is my order A17?"))

langfuse.flush()
local_langfuse.tree()
print(local_langfuse.REQUESTS)
generation = next(span for span in local_langfuse.SPANS if span["name"] == "write-reply")
print(generation["attributes"]["langfuse.observation.usage_details"])

python openai_desk.py

The chat call became a generation under the agent without any Langfuse code around it. REQUESTS shows the OpenAI client's request going to the local server, then the traces export.

Usage arrived in OpenAI's own field names, prompt_tokens and completion_tokens. Langfuse maps these to its input and output when it stores them. The wrapper also recorded the full message list as input and the default model parameters, so everything in the prompt, including a customer's text, is part of the trace.