What should be versioned in an LLM application?

30-second answerSay your answer out loud first, then reveal.

Everything that affects behaviour: application code; prompts and prompt templates; model identifiers (exact versions, not aliases) and generation parameters; tool and function schemas; retrieval configuration (chunking, embedding model, index version, reranker, top-k); guardrail configs and thresholds; agent graphs and workflows; fine-tuned model weights or adapters and their training data; eval datasets and judge prompts. Each production request should log the versions used, so any output can be reproduced and debugged.

A "release" for an LLM feature = a bundle

yaml

feature: claims-summary
release: 2026.10.06-3
code_commit: a1b2c3d
prompt: claims_summary@v14
model: provider-x/model-y-2026-08-15   # pinned, not "latest"
params: {temperature: 0.2, max_tokens: 800}
tools_schema: claims_tools@v5
retrieval: {index: claims_docs@2026-10-01, embedder: emb-v3, top_k: 6, reranker: rr-v2}
guardrails: claims_guardrails@v3
eval_dataset: claims_golden@v9
eval_results: run_8841 (passed gates)

Why it matters

Reproducibility: debugging "why did it say this on Tuesday?" needs the exact configuration.
Rollback: you can revert just the prompt, or just the index.
Auditability: regulated industries require knowing what produced each decision.
Comparison: eval results tied to a specific bundle.

Common mistakes

Using model aliases like "latest" in production.
Prompts edited directly in a UI or database without version history.

PreviousDescribe the lifecycle of an LLM application in production.

Slow is fine. Stopping is the only problem.

What should be versioned in an LLM application?

A "release" for an LLM feature = a bundle

Why it matters

Common mistakes

Related