LiteLLMLiteLLM 1.105 · Python 3.10+
Dashboard
0%

Tokens and cost

Token usage and cost are fields LiteLLM puts on every response: usage counts the tokens, and _hidden_params["response_cost"] holds the price of the call in US dollars.

Last updated: 12 Oct, 2026 · LiteLLM 1.105

Streaming: the answer a piece at a time rebuilt a response and read its cost without explaining it. This lesson is about cost. It shows where the number comes from and how to read it. It also shows one function that looks like the way to get it and fails for the course's models.

Reading the usage and the cost of a call

response.usage holds three counts: prompt_tokens for what was sent, completion_tokens for what came back and total_tokens for both. LiteLLM works out the price from those counts and its table of prices, and puts it in the response's hidden parameters. The LiteLLM documentation names response._hidden_params["response_cost"] as the place to read it.

ExampleAPI keyTokens and cost for two providers
from litellm import completion

messages = [{"role": "user", "content": "Name one colour."}]

for model in ["groq/openai/gpt-oss-120b", "gemini/gemini-2.5-flash-lite"]:
    response = completion(model=model, messages=messages, max_tokens=400)
    usage = response.usage
    cost = response._hidden_params["response_cost"]
    print(model, usage.prompt_tokens, usage.completion_tokens, usage.total_tokens, cost)
  • Each line shows the model, then prompt tokens, completion tokens, the total and the cost in dollars.
  • The cost is a very small decimal, a fraction of a cent. It is the price of one short call, and it adds up over thousands of calls.
  • The same field worked for both providers, which is the reason to read the cost from one place.

Working the cost out from the price table

LiteLLM keeps a table of prices, litellm.model_cost, with a price per input token and a price per output token for each model it knows. The next example reads both prices for the Groq model and rebuilds the cost by hand from the usage.

ExampleAPI keyThe cost of one call, worked out by hand and as reported
import litellm

model = "groq/openai/gpt-oss-120b"
messages = [{"role": "user", "content": "Name one colour."}]
response = litellm.completion(model=model, messages=messages, max_tokens=400)

prices = litellm.model_cost[model]
usage = response.usage
by_hand = (usage.prompt_tokens * prices["input_cost_per_token"]
           + usage.completion_tokens * prices["output_cost_per_token"])
reported = response._hidden_params["response_cost"]

print("in the price table:", model in litellm.model_cost)
print("input price per token: ", prices["input_cost_per_token"])
print("output price per token:", prices["output_cost_per_token"])
print("tokens:", usage.prompt_tokens, "in,", usage.completion_tokens, "out")
print(f"cost by hand:  ${by_hand:.8f}")
print(f"cost reported: ${reported:.8f}")
print("they match:", round(by_hand, 10) == round(reported, 10))
  • The model is in the table. The first line is True, so LiteLLM knows this model's price.
  • The two prices are per token. An output token costs more than an input token.
  • The cost of the call is printed in dollars, twice. The by-hand figure is prompt tokens times the input price, plus completion tokens times the output price. The reported figure is the one LiteLLM put on the response.
  • The last line is True, because the two figures agree. The cost is read from the response, and it is the same number the price table gives.

Why completion_cost() raises for these models

LiteLLM also has a function named completion_cost(), and it is the obvious way to ask for a cost. For the course's Groq models it fails. This call passes the response.

ExampleAPI keycompletion_cost() on a Groq gpt-oss response
import litellm

messages = [{"role": "user", "content": "Name one colour."}]
response = litellm.completion(model="groq/openai/gpt-oss-120b", messages=messages, max_tokens=400)

litellm.completion_cost(completion_response=response)

Passing the model name as well does not help.

ExampleAPI keycompletion_cost() with the model passed in
import litellm

messages = [{"role": "user", "content": "Name one colour."}]
response = litellm.completion(model="groq/openai/gpt-oss-120b", messages=messages, max_tokens=400)

litellm.completion_cost(completion_response=response, model="groq/openai/gpt-oss-120b")

Reading the ModelNotMappedError

  • The model is mapped, and the function still says it is not. The earlier example proved groq/openai/gpt-oss-120b is in litellm.model_cost.
  • The cause is the slash inside the model id. The id is openai/gpt-oss-120b. The function removes the groq/ prefix and then reads openai as the provider, so it looks for a model called gpt-oss-120b under OpenAI and finds none. The error message shows custom_llm_provider=openai for that reason.
  • Any model id with its own slash can hit this, whatever the provider. The call itself succeeded, and only the separate price lookup failed.

The call's own _hidden_params["response_cost"] does not go through that lookup, and it holds the right figure, as the first two examples showed. Use it. Callbacks see the same number as kwargs["response_cost"], which Callbacks: tracking every call reads.

Estimating the input cost before sending

token_counter counts the tokens of a message on your machine. Multiplied by the input price from the table it estimates what the prompt alone will cost, before any call is made.

ExampleAn estimate before sending
import litellm

model = "groq/openai/gpt-oss-120b"
messages = [{"role": "user", "content": "I was charged twice for one order. Please refund the extra payment."}]

tokens = litellm.token_counter(model=model, messages=messages)
price = litellm.model_cost[model]["input_cost_per_token"]
print(tokens, "prompt tokens")
print(f"{tokens * price:.8f} dollars for the prompt")

It is an estimate. The provider counts tokens its own way, and the answer's tokens are not included, so the figure is a floor for the cost of the call.

Reading cost from the response vs completion_cost()

response._hidden_params["response_cost"]litellm.completion_cost()
Works for groq/openai/gpt-oss-120bYesNo, raises ModelNotMappedError
Needs the responseYes, it is on itYes, or a prompt and a completion
Works for GeminiYesCheck for each model before relying on it

Where you use cost tracking

  • A running total per team or per customer, added up in a callback.
  • Choosing a cheaper model by comparing the cost of the same prompt on two models.
  • A budget check that stops a loop when the total passes a limit.
Watch out. A cost of None or a ModelNotMappedError means the price could not be looked up, not that the call was free. Check model in litellm.model_cost and read the cost from the response itself.
Try it yourself
  • Add "groq/openai/gpt-oss-20b" to the first loop and compare its cost with the 120b model for the same prompt.
  • Print litellm.model_cost["groq/openai/gpt-oss-20b"] and read its two prices.
  • Multiply the cost of one call by 10,000 to see what a day of tickets would cost at that rate.

This is what real progress feels like.