Tokens and cost
Token usage and cost are fields LiteLLM puts on every response: usage counts the tokens, and _hidden_params["response_cost"] holds the price of the call in US dollars.
Last updated: 12 Oct, 2026 · LiteLLM 1.105
Streaming: the answer a piece at a time rebuilt a response and read its cost without explaining it. This lesson is about cost. It shows where the number comes from and how to read it. It also shows one function that looks like the way to get it and fails for the course's models.
Reading the usage and the cost of a call
response.usage holds three counts: prompt_tokens for what was sent, completion_tokens for what came back and total_tokens for both. LiteLLM works out the price from those counts and its table of prices, and puts it in the response's hidden parameters. The LiteLLM documentation names response._hidden_params["response_cost"] as the place to read it.
from litellm import completion
messages = [{"role": "user", "content": "Name one colour."}]
for model in ["groq/openai/gpt-oss-120b", "gemini/gemini-2.5-flash-lite"]:
response = completion(model=model, messages=messages, max_tokens=400)
usage = response.usage
cost = response._hidden_params["response_cost"]
print(model, usage.prompt_tokens, usage.completion_tokens, usage.total_tokens, cost)groq/openai/gpt-oss-120b 75 32 107 3.0449999999999998e-05 gemini/gemini-2.5-flash-lite 5 1 6 9e-07
- Each line shows the model, then prompt tokens, completion tokens, the total and the cost in dollars.
- The cost is a very small decimal, a fraction of a cent. It is the price of one short call, and it adds up over thousands of calls.
- The same field worked for both providers, which is the reason to read the cost from one place.
Working the cost out from the price table
LiteLLM keeps a table of prices, litellm.model_cost, with a price per input token and a price per output token for each model it knows. The next example reads both prices for the Groq model and rebuilds the cost by hand from the usage.
import litellm
model = "groq/openai/gpt-oss-120b"
messages = [{"role": "user", "content": "Name one colour."}]
response = litellm.completion(model=model, messages=messages, max_tokens=400)
prices = litellm.model_cost[model]
usage = response.usage
by_hand = (usage.prompt_tokens * prices["input_cost_per_token"]
+ usage.completion_tokens * prices["output_cost_per_token"])
reported = response._hidden_params["response_cost"]
print("in the price table:", model in litellm.model_cost)
print("input price per token: ", prices["input_cost_per_token"])
print("output price per token:", prices["output_cost_per_token"])
print("tokens:", usage.prompt_tokens, "in,", usage.completion_tokens, "out")
print(f"cost by hand: ${by_hand:.8f}")
print(f"cost reported: ${reported:.8f}")
print("they match:", round(by_hand, 10) == round(reported, 10))in the price table: True input price per token: 1.5e-07 output price per token: 6e-07 tokens: 75 in, 40 out cost by hand: $0.00003525 cost reported: $0.00003525 they match: True
- The model is in the table. The first line is
True, so LiteLLM knows this model's price. - The two prices are per token. An output token costs more than an input token.
- The cost of the call is printed in dollars, twice. The by-hand figure is prompt tokens times the input price, plus completion tokens times the output price. The reported figure is the one LiteLLM put on the response.
- The last line is
True, because the two figures agree. The cost is read from the response, and it is the same number the price table gives.
Why completion_cost() raises for these models
LiteLLM also has a function named completion_cost(), and it is the obvious way to ask for a cost. For the course's Groq models it fails. This call passes the response.
import litellm
messages = [{"role": "user", "content": "Name one colour."}]
response = litellm.completion(model="groq/openai/gpt-oss-120b", messages=messages, max_tokens=400)
litellm.completion_cost(completion_response=response)Traceback (most recent call last):
File "main.py", line 6, in <module>
litellm.completion_cost(completion_response=response)
litellm.exceptions.ModelNotMappedError: This model isn't mapped yet. model=openai/gpt-oss-120b, custom_llm_provider=openai. Add it here - https://github.com/BerriAI/litellm/blob/main/model_prices_and_context_window.json.Passing the model name as well does not help.
import litellm
messages = [{"role": "user", "content": "Name one colour."}]
response = litellm.completion(model="groq/openai/gpt-oss-120b", messages=messages, max_tokens=400)
litellm.completion_cost(completion_response=response, model="groq/openai/gpt-oss-120b")Traceback (most recent call last):
File "main.py", line 6, in <module>
litellm.completion_cost(completion_response=response, model="groq/openai/gpt-oss-120b")
litellm.exceptions.ModelNotMappedError: This model isn't mapped yet. model=groq/openai/gpt-oss-120b, custom_llm_provider=openai. Add it here - https://github.com/BerriAI/litellm/blob/main/model_prices_and_context_window.json.Reading the ModelNotMappedError
- The model is mapped, and the function still says it is not. The earlier example proved
groq/openai/gpt-oss-120bis inlitellm.model_cost. - The cause is the slash inside the model id. The id is
openai/gpt-oss-120b. The function removes thegroq/prefix and then readsopenaias the provider, so it looks for a model calledgpt-oss-120bunder OpenAI and finds none. The error message showscustom_llm_provider=openaifor that reason. - Any model id with its own slash can hit this, whatever the provider. The call itself succeeded, and only the separate price lookup failed.
The call's own _hidden_params["response_cost"] does not go through that lookup, and it holds the right figure, as the first two examples showed. Use it. Callbacks see the same number as kwargs["response_cost"], which Callbacks: tracking every call reads.
Estimating the input cost before sending
token_counter counts the tokens of a message on your machine. Multiplied by the input price from the table it estimates what the prompt alone will cost, before any call is made.
import litellm
model = "groq/openai/gpt-oss-120b"
messages = [{"role": "user", "content": "I was charged twice for one order. Please refund the extra payment."}]
tokens = litellm.token_counter(model=model, messages=messages)
price = litellm.model_cost[model]["input_cost_per_token"]
print(tokens, "prompt tokens")
print(f"{tokens * price:.8f} dollars for the prompt")21 prompt tokens 0.00000315 dollars for the prompt
It is an estimate. The provider counts tokens its own way, and the answer's tokens are not included, so the figure is a floor for the cost of the call.
Reading cost from the response vs completion_cost()
| response._hidden_params["response_cost"] | litellm.completion_cost() | |
|---|---|---|
Works for groq/openai/gpt-oss-120b | Yes | No, raises ModelNotMappedError |
| Needs the response | Yes, it is on it | Yes, or a prompt and a completion |
| Works for Gemini | Yes | Check for each model before relying on it |
Where you use cost tracking
- A running total per team or per customer, added up in a callback.
- Choosing a cheaper model by comparing the cost of the same prompt on two models.
- A budget check that stops a loop when the total passes a limit.
None or a ModelNotMappedError means the price could not be looked up, not that the call was free. Check model in litellm.model_cost and read the cost from the response itself.Related
- Previous: Streaming: the answer a piece at a time
- Next: Exceptions: one set for every provider
- Reference: Completion token usage and cost in the LiteLLM docs
- Add
"groq/openai/gpt-oss-20b"to the first loop and compare its cost with the 120b model for the same prompt. - Print
litellm.model_cost["groq/openai/gpt-oss-20b"]and read its two prices. - Multiply the cost of one call by 10,000 to see what a day of tickets would cost at that rate.
This is what real progress feels like.