1
Curious builder0 XP earned · 300 to level 2
0 daysFinish a lesson to begin
Badge collection0 of 6 unlocked
51 small wins to finish your pathNext question →
Design an AI reply suggestion feature (smart replies) for a customer-support inbox used by 5,000 agents.
30-second answerSay your answer out loud first, then reveal.
Requirements to clarify: languages, ticket volume (e.g. 200K tickets/day), latency (draft ready when the agent opens the ticket), allowed actions (refund promises?), tone guidelines.
Architecture
- Trigger: on a new customer message, an event goes to a queue and a worker pre-generates the draft (so it's ready before the agent opens the ticket). On-demand regenerate with instructions ("more formal", "offer replacement").
- Context assembly: last N messages, customer tier and order status (API), top-3 KB articles (RAG), relevant macros, company tone guide.
- Generation: mid-size model; structured output
{reply, cited_articles, suggested_actions}. - Guardrails: no promises outside policy (refund amounts), no internal notes leaked, PII rules, language match.
- UI: draft in the composer, citations visible, one-click insert, inline edit.
- Feedback: accepted unchanged / edited (edit distance) / discarded, logged with the trace ID.
Metrics
| Metric | Purpose |
|---|---|
| Acceptance rate, edit distance | Draft quality |
| Handle time per ticket | Business impact |
| CSAT, reopen rate | Ensure quality didn't drop |
| Cost per draft | Economics (skip drafts for tickets closed by automation) |
Improvement loop: agents' edited final replies become training and eval data (with consent), giving per-team style adaptation via few-shot retrieval of similar past replies.
Related
This is what real progress feels like.