LLMOpsThe LLMOps lifecycle · MLflow 3 · FastAPI · Docker · AWS/Azure
Dashboard
0%
1
Curious builder0 XP earned · 300 to level 2
0 daysFinish a lesson to begin
Badge collection0 of 6 unlocked
41 small wins to finish your pathNext lesson →

Choosing an LLM: providers and model cards

Choosing an LLM is the model-selection stage: you weigh hosted providers against open-source models and read each candidate's model card for size, context window, cost and licence before you commit.

Last updated: 15 Oct, 2026 · LLMOps

Data for fine-tuning and the The RAG data pipeline decided how the model gets its knowledge. Model selection decides which model that is. The Detailed LLMOps Project Lifecycle video calls this out as its own stage: try several models, and read their numbers before picking one.

Choosing an LLM and reading model cards · from the Detailed LLMOps Project Lifecycle video · 5:18 to 6:40

Hosted providers or open-source

The first split is between calling a model through a provider API and running an open-source model yourself. The video names OpenAI, Anthropic and Google Gemini as providers you can try, and Hugging Face as where the open-source models and their details live.

  • A provider API (OpenAI, Anthropic, Google Gemini) is a call away, kept current and scaled for you, and billed per token.
  • An open-source model from Hugging Face you host yourself, which you can fine-tune and run in your own cloud for privacy, at the cost of running the serving infrastructure.

What a model card tells you

A model card is the spec sheet a model ships with. The video points at Hugging Face as the place to read the architecture, the size and the other numbers that decide whether a model fits. Four of those numbers do most of the deciding.

  • Size, the parameter count, which sets how much hardware it takes to run and often how capable it is.
  • Context window, how many tokens of input and output it can hold at once, which caps how much you can put in a prompt or retrieve into it.
  • Cost, the price per token for a hosted model or the hardware to serve an open one.
  • Licence, what you are allowed to do with an open model, including whether you may use it commercially.
Hosted providerSelf-hosted open model
You get it fromAn API (OpenAI, Anthropic, Gemini)Hugging Face, on your own hardware
You pay forTokens, per callThe hardware that serves it
Fine-tune freelyOnly what the provider allowsYes, it is your model
Data leaves your systemsYes, to the providerNo, if you host it
You operateThe prompt and the costThe whole serving stack
Two panels on where the model runs. With a hosted provider API, your app calls the provider across a dashed red boundary marked your data leaves here; you pay per token and operate the prompt and the cost, not the serving. With a self-hosted open model, the app and the model sit inside a green dashed box marked nothing leaves your systems; you pay for the hardware, operate the whole serving stack and may fine-tune freely. Below, the four model-card fields to read first: size, context window, cost and licence.

How to pick

The lifecycle video's advice is plain: try several, then choose on the numbers and the constraints, not on reputation.

  • Start from the task and the constraints. A privacy rule that forbids data leaving your systems rules out a hosted API before any quality test.
  • Try more than one. Run the same evaluation set against each candidate and compare, rather than committing to the first that works.
  • Read the card before the benchmark. Context window and licence can disqualify a model no matter how well it scores.

Where model selection bites

  • A context window too small for the chunks you retrieve, found only after the RAG pipeline is built.
  • A licence that forbids commercial use, found after a model is already in the product.
  • A per-token cost that is fine in a demo and ruinous at production traffic.
Watch out. The model you pick is not a permanent choice, so keep it swappable. Providers release new models and retire old ones, and prices change, so route model calls through one place in your code and record which model served each result. Controlling the cost of API calls returns to this when the bill arrives.
Try it yourself
  • Pick two candidate models for your task, one hosted and one open-source, and fill in the size, context window, cost and licence for each from their model cards.
  • Find the context window of your chosen model and work out how many retrieved chunks you could fit alongside a question.
  • Check the licence of one open model on Hugging Face and confirm whether it allows the use you have in mind.
PreviousData for fine-tuning

Little by little, you're building something great.