What are the main options for deploying LLMs, and how do you choose?

30-second answerSay your answer out loud first, then reveal.

(1) Third-party model APIs (fastest, strongest models, pay per token). (2) Managed cloud model platforms (models served inside your cloud account and region, with enterprise controls). (3) Self-hosted open models on your own GPUs, via Kubernetes with vLLM, SGLang, TGI or TensorRT-LLM (full control, data stays in-house, fixed GPU cost). (4) Edge or on-device small models (offline, low latency, privacy). Choose by capability needs, data constraints, volume and cost, latency and team skills. Many companies mix them.

Option	Pros	Cons	Good for
Model provider API	Best models, zero infra, fast iteration	Data leaves network (with contractual controls), rate limits, vendor dependency	Most new features, complex reasoning
Managed cloud platform (in your cloud account)	Regional processing, IAM integration, consolidated billing	Model selection varies by region; some lag	Enterprises with cloud commitments and residency needs
Self-hosted (Kubernetes + inference server)	Control, privacy, predictable cost at high utilisation, fine-tuned models	GPU ops, capacity planning, on-call	High-volume narrow tasks, sensitive data, custom models
Edge / on-device	Offline, privacy, no per-token cost	Small models only, device fragmentation, update logistics	Mobile features, field devices

Decision inputs: eval results per option, data classification, monthly token volume vs GPU cost at realistic utilisation, latency SLOs, team capability, and lock-in tolerance. Use a gateway abstraction so you can switch or mix.

PreviousHow do dev, staging and production environments work for LLM applications? What's tricky about them?

Every expert started right here.

What are the main options for deploying LLMs, and how do you choose?

Related