Explain vector quantization and Matryoshka embeddings. What are the trade-offs?

30-second answerSay your answer out loud first, then reveal.

Quantization stores vectors with fewer bits: int8 (about 4x smaller than float32), binary (1 bit per dimension, about 32x smaller), or product quantization (compressed codes). Matryoshka embeddings are trained so the first N dimensions are a usable embedding on their own, so you can truncate 3072 → 256 dims. Both cut memory and speed up search at some cost to accuracy, which you recover by over-fetching candidates and rescoring with full-precision vectors.

Quantization types

Type	Size vs float32	Accuracy impact	Notes
float16	2x smaller	Negligible	Easy win
int8 (scalar)	4x smaller	Small	Widely supported
Binary	32x smaller	Noticeable alone	Use Hamming distance for candidates, then rescore with full vectors
Product quantization (PQ)	8–64x	Moderate	Classic in FAISS / IVF-PQ

Rescoring pattern

Search the quantized index for the top 200 candidates (fast, in RAM).
Load the full-precision vectors for those 200 (from disk) and recompute exact similarity.
Return the top 10. This recovers most of the lost accuracy.

Matryoshka Representation Learning (MRL)

Training puts the most important information in the leading dimensions.
Truncate to 256 or 512 dims for a cheap first-stage search; use full dims for rescoring (an adaptive retrieval funnel).
Supported by several modern embedding models (e.g. OpenAI text-embedding-3 via its dimensions parameter, plus several open models).

When to use: large corpora (tens of millions of vectors and up), memory-constrained deployments, cost pressure. Always measure the recall drop on your eval set.

PreviousWhat drives the cost of a RAG system, and how do you reduce it?

Little by little, you're building something great.

Explain vector quantization and Matryoshka embeddings. What are the trade-offs?

Quantization types

Rescoring pattern

Matryoshka Representation Learning (MRL)

Related