Design a system that sends AI-personalised notifications to 100M users daily.
30-second answerSay your answer out loud first, then reveal.
Why not per-user LLM generation at send time
- 100M calls/day means high cost, a latency and throughput bottleneck, and an unreviewable risk (one bad generation goes out to millions).
- Personalisation value usually comes from what and when (targeting), more than from bespoke wording.
Where per-user generation can make sense: smaller, high-value segments, run in batch with validation (e.g. weekly personalised learning progress summaries), using templates for structure.
Safety and compliance: no fabricated discounts or claims (prices and offers come from data, never generated), regulatory consent (opt-in), language and cultural review, unsubscribe and complaint monitoring as guardrail metrics.
Metrics: CTR, conversion, unsubscribe rate, complaint rate, long-term retention (avoid notification fatigue).
Related
This is what real progress feels like.