Open-weight vs closed (API) models: what are the trade-offs?

30-second answerSay your answer out loud first, then reveal.

Closed API models (from OpenAI, Anthropic, Google) generally offer the strongest capabilities with no infrastructure to run, but you get less control, depend on a vendor, and send data to a third party. Open-weight models (Llama, Qwen, Mistral, DeepSeek, Gemma and others) can be self-hosted, fine-tuned and run fully on-prem with predictable behaviour, but you own the serving, scaling, safety and updates. The choice depends on capability needs, data constraints, cost at your volume, and team skills.

Factor	Closed API	Open-weight (self-hosted)
Peak capability	Usually highest	Strong and closing the gap
Setup	Minutes	GPUs, serving stack, MLOps
Data control	Data leaves your network (though enterprise terms help)	Full control; air-gapped possible
Customisation	Prompting; limited fine-tuning	Full fine-tuning, any modification
Cost profile	Pay per token; great at low or variable volume	Fixed GPU cost; cheaper at high, steady volume
Version stability	Models deprecated on vendor timelines	You choose when to upgrade
Latency control	Shared infrastructure	Tunable; on-prem or edge
Licence	Terms of service	Varies; check commercial use and restrictions

"Open source" vs "open weight": many models release weights but not training data or code, and some have usage restrictions in their licences. Be precise in interviews.

Common hybrid: an API model for complex reasoning, plus a self-hosted small model for high-volume simple tasks, sensitive data, or offline use.

Follow-ups to expect

How would you calculate the break-even between API and self-hosting? (Q48.)

PreviousWhat does "7B" or "70B parameters" mean, and how much memory does a model need?

This is what real progress feels like.

Open-weight vs closed (API) models: what are the trade-offs?

Follow-ups to expect

Related