Design a self-hosted inference cluster on Kubernetes serving five open models (chat, code, embeddings, reranker, small classifier) with SLOs.

Use separate GPU pools sized per workload class: large chat and code models on multi-GPU high-memory nodes with tensor parallelism, and the embeddings, reranker and classifier on smaller GPUs or MIG slices with high batch throughput. Put an inference gateway in front (routing, auth, quotas, load- and cache-aware balancing), model servers per family (vLLM/SGLang for generation, an embedding/reranker server such as TEI or Triton for encoders), autoscaling on queue and KV metrics, weight caching, observability, and capacity planning against per-model SLOs.