Cost: counting tokens before you pay

Hosted models charge per million tokens, one price for tokens sent and another for tokens written. With a token count, cost is arithmetic done before sending.

The examples below reuse structured from lesson 10 and examples from lesson 11.

def cost(input_tokens, output_tokens, input_price, output_price):
    return (input_tokens * input_price + output_tokens * output_price) / 1_000_000

Prices are quoted per million tokens, hence the division. The prices below are round example numbers; look up the current ones for the model you use, because they change often.

messages = [{"role": "system", "content": structured}, *examples, {"role": "user", "content": "My parcel has not arrived"}]
input_tokens = len(tokenizer.apply_chat_template(messages, tokenize=True, add_generation_prompt=True, return_dict=True)["input_ids"])
output_tokens = len(tokenizer.encode('{"category": "shipping", "priority": 3}'))
print(input_tokens, "in,", output_tokens, "out")

per_ticket = cost(input_tokens, output_tokens, input_price=0.50, output_price=1.50)
print(f"${per_ticket:.6f} per ticket")
print(f"${per_ticket * 50_000:.2f} for 50,000 tickets a month")

Nearly all of the cost is input: the system prompt and examples are sent again with every ticket, while the answer is a dozen tokens. Output tokens usually cost several times more than input, so a task that writes long answers flips this.

Examples are not free. The three examples that raised accuracy in lesson 13 are about half of every call's tokens. That is the trade: measure how much accuracy they buy against what they cost.

Different models, different tokens

import tiktoken

openai_tokenizer = tiktoken.get_encoding("o200k_base")
for text in ["My parcel has not arrived", "1234567", "नमस्ते, मेरा पार्सल नहीं आया", "我的包裹还没有到"]:
    print(len(tokenizer.encode(text)), "Qwen tokens,", len(openai_tokenizer.encode(text)), "OpenAI tokens")

o200k_base is the tokenizer OpenAI's recent models use. Plain English comes out the same here, but numbers, Hindi and Chinese do not, and neither tokenizer is cheaper for everything. The same text gives different counts under different tokenizers, so count with the tokenizer of the model you will pay for. Providers also report the exact counts in each response, like the usage field in APIs for AI.