How do you manage model weights and artifacts in production?

30-second answerSay your answer out loud first, then reveal.

Store weights in a model registry or object storage with immutable versions, metadata (source, licence, training data lineage, eval results, checksums) and access control. Prefer safe formats (safetensors over pickle-based formats, which can execute code on load), verify integrity with checksums or signatures, cache weights close to GPUs (local NVMe, a pre-warmed volume, or a regional cache) for fast startup, and manage lifecycle (promotion stages, deprecation, deletion).

Practices

Concern	Approach
Registry	MLflow / cloud model registry / an object store with a metadata DB; stages: candidate → staging → production → archived
Provenance	Base model + version, licence, fine-tuning data version, training config, eval report
Security	safetensors; scan third-party models; signed artifacts; restricted write access
Integrity	SHA-256 checksums verified at download
Performance	Parallel and streaming download; local NVMe cache; shared read-only volumes; pre-loading on node startup
Size management	Quantized variants stored separately (FP8, AWQ, GGUF) with their own eval results
Adapters	LoRA adapters versioned separately, linked to their base model version
Lifecycle	Retention policy for old checkpoints; keep the rollback version available

Supply-chain risk: downloading arbitrary models from public hubs into production without review is like running unvetted code. Pin exact revisions and review licences (commercial-use terms vary).

PreviousWhat metrics should you autoscale self-hosted LLM servers on, and why not CPU?

Little by little, you're building something great.

How do you manage model weights and artifacts in production?

Practices

Related