During a pilot, users complain the assistant is "too slow." You're on a call with the customer. How do you diagnose it quickly?

30-second answerSay your answer out loud first, then reveal.

Get specifics: which action, how slow, for whom, since when. Check traces for those requests to see the latency breakdown per stage (network, retrieval, LLM time to first token, generation length, tool calls, retries). Identify the dominant contributor, then apply targeted fixes: streaming, shorter prompts or outputs, a faster model for simple steps, parallel calls, caching, removing sequential steps, or fixing slow customer APIs. Communicate a short-term mitigation and a follow-up plan.

On the call

Clarify: "Is it slow to start responding, or slow to finish? For all requests or certain types? Since when?"
Look at traces live (if you have observability): pick 3–5 slow requests.

Common findings and fixes

Finding	Fix
No streaming, so users wait 12s for the full answer	Enable streaming; show progress
Customer CRM API calls take 4s each, sequentially	Parallelise; cache; ask their IT about indexes
Prompt has 30K tokens of context	Trim retrieval; summarise history
Agent does 8 sequential LLM calls	Merge steps; use a workflow for predictable parts
Slow only at 10am peak	Rate limiting / queueing; capacity
Slow only for remote office users	Network path / region; deploy closer
Retries due to JSON errors	Structured outputs

Communicate: "The main delay is the CRM lookup (60% of the time). Today we'll enable streaming and caching for customer profiles; with your IT team we'll look at the slow endpoint this week."

Interview signal: structured triage, data before guesses, and calm communication.

PreviousThe customer keeps adding requirements mid-project ("can it also do X?"). How do you manage scope creep?

Every expert started right here.

During a pilot, users complain the assistant is "too slow." You're on a call with the customer. How do you diagnose it quickly?

On the call

Common findings and fixes

Related