Masking data before it leaves

mask_otel_spans runs on each batch right before export and can replace or delete any attribute. If it raises, the SDK drops the whole batch rather than send it unmasked.

Lesson 14 left a card number in the agent's input and the model's messages. Langfuse's masking hook sees every observation this client exports, from the SDK or from other OpenTelemetry tools, after everything else has run and before anything is sent.

import re

from langfuse.types import MaskOtelSpansResult, OtelSpanPatch

CARD = re.compile(r"\b\d(?:[ -]?\d){12,15}\b")


def mask_cards(*, params):
    patches = {}
    for span_id, span in params.spans.items():
        changed = {key: CARD.sub("[card]", value) for key, value in span.attributes.items() if CARD.search(value)}
        if changed:
            patches[span_id] = OtelSpanPatch(set_attributes=changed)
    return MaskOtelSpansResult(span_patches=patches)

The hook is called with a batch of observations, keyed by an identifier. It returns patches only for the ones that change: here, every attribute whose text contains something shaped like a card number, with the number replaced.

from langfuse import Langfuse, propagate_attributes

import local_langfuse
from desk import answer

url = local_langfuse.start()
langfuse = Langfuse(public_key="pk-lf-local", secret_key="sk-lf-local", base_url=url, mask_otel_spans=mask_cards)

with propagate_attributes(tags=["billing"]):
    print(answer("My card 4111 1111 1111 1111 was charged twice for A17"))

langfuse.flush()
print(len(local_langfuse.SPANS), "observations received")
for span in local_langfuse.SPANS:
    print(span["name"], "|", span["attributes"].get("langfuse.observation.input"))

python masking.py

Nothing arrived. The log line says why: the hook raised, and the SDK dropped the batch. The tags from propagate_attributes arrive in the hook as a tuple of strings, and CARD.search on a tuple raises TypeError. Dropping the batch is the safe failure, because the alternative would be sending it unmasked, but it means a bug in your hook costs you traces silently.

def mask_cards(*, params):
    patches = {}
    for span_id, span in params.spans.items():
        changed = {key: CARD.sub("[card]", value) for key, value in span.attributes.items() if isinstance(value, str) and CARD.search(value)}
        if changed:
            patches[span_id] = OtelSpanPatch(set_attributes=changed)
    return MaskOtelSpansResult(span_patches=patches)

python masking.py

With the type check, all three observations arrived, and both inputs that held the card now hold [card]. Your program still printed the model's answer normally: masking changes what is exported, never what your code sees.

What the hook can and cannot do

• It can set and delete attributes. It cannot change names, ids or parents.
• An invalid patch for one observation drops only that observation; a hook that raises drops the batch.
• It runs on the exporter's background thread, or during flush on yours, so it has to be fast and must not depend on the request being handled.
• It only masks what this Langfuse client exports. Another OpenTelemetry exporter in the same program sends its own unmasked copy.

The older mask argument still works: a function that changes each input, output and metadata value when the SDK records it. It never sees attributes set by third-party OpenTelemetry libraries, such as a database driver's, so mask_otel_spans is the one to reach for in new code.