A customer asks, "How much will this cost us per month in model usage?" How do you estimate it?

30-second answerSay your answer out loud first, then reveal.

Estimate per-transaction token usage from the actual pipeline (prompt size including retrieved context, output length, number of LLM calls per task including retries and agent steps), multiply by expected volume and model prices, and add embeddings, re-indexing and infrastructure. Validate with logs from the pilot, give a range, and explain the levers that lower cost (smaller models for some steps, caching, batch processing).

Method

Measure per task from pilot traces: average input tokens, output tokens, calls per task, e.g. 3 calls × (4,000 in + 400 out).
Volume: tasks per month at current and projected scale.
Prices: per model used at each step (they differ for input, output and cached input).
Other costs: embeddings for new documents, vector DB, hosting, monitoring.
Contingency: +20–30% for retries, growth and longer documents.

Example

text

Per task: 12,000 input tokens + 1,200 output tokens (3 calls)
Volume: 100,000 tasks/month
Input:  1.2B tokens × price_in
Output: 120M tokens × price_out
→ present as a range across 2-3 model options and caching assumptions

Levers to mention: routing (a small model for classification), prompt caching for static instructions, reducing retrieved context, batch APIs for non-urgent work, and limits on agent steps.

Interview signal: grounded estimation from real traces, a transparent range, and cost-control levers, rather than a guess.

PreviousHow do you estimate and present ROI for an AI project to a customer?

Little by little, you're building something great.

A customer asks, "How much will this cost us per month in model usage?" How do you estimate it?

Method

Example

Related