How do you design an AI system to meet data residency and multi-region requirements?

Design elements

1. Tenant → region mapping at sign-up; all requests routed to that region's stack (DNS / gateway routing).
2. Region-local data stores: databases, object storage, vector indexes, caches, logs and traces, backups.
3. Model access: in-region API endpoints from providers or cloud marketplaces, or self-hosted open models where the provider has no in-region capacity.
4. Global control plane, regional data plane: configuration, prompt registry and feature flags can be global; customer content stays regional.
5. Failover constraints: disaster recovery only to regions permitted for that tenant (e.g. two Indian regions).
6. Observability without content leakage: export metrics globally, keep raw prompts and responses in-region.
7. Evals: run model quality evals per region if different models are used in different regions.

Trade-offs: higher cost (duplicate infrastructure), some regions lack the latest models, operational complexity, and model behaviour may differ across regions.

Related

• Previous: Q31. How do you protect sensitive data (PII) when using third-party LLM APIs?
• Next: Q33. Users say the assistant "got worse" over the last few weeks, but nobody changed anything. How do you investigate, and how do you detect this earlier next time?