What is late interaction (ColBERT), and how does it compare to bi-encoders and cross-encoders?

30-second answerSay your answer out loud first, then reveal.

ColBERT stores one embedding per token for each document instead of one per chunk. At query time each query token finds its most similar document token (MaxSim), and the scores are summed. This captures fine-grained term-level matching, getting close to cross-encoder quality, while document token embeddings are still pre-computed, so it's much faster than a cross-encoder. The cost is a much larger index.

Bi-encoder vs late interaction vs cross-encoder

	Bi-encoder	Late interaction (ColBERT)	Cross-encoder
Representation	1 vector per chunk	1 vector per token	No stored vectors; joint encoding
Precompute docs	Yes	Yes	No
Query-time cost	Lowest	Medium	Highest
Quality	Good	Better	Best
Index size	Small	Large (tens of times bigger)	None
Typical use	First-stage retrieval	First stage or reranking	Reranking top-N

MaxSim scoring

text

score(q, d) = Σ over query tokens i of  max over doc tokens j of  (q_i · d_j)

Practical notes

ColBERTv2 / PLAID reduce index size with compression and centroids.
Often used as a reranker over BM25 or dense candidates, which avoids the full index cost.
The same idea extends to images: ColPali applies late interaction over image patches of document pages (Q34).

When to mention it: when asked how to improve precision beyond a bi-encoder without paying full cross-encoder latency.

PreviousRetrieved documents contradict each other (an old vs new policy, two teams' docs). How should the system handle it?

Slow is fine. Stopping is the only problem.

What is late interaction (ColBERT), and how does it compare to bi-encoders and cross-encoders?

Bi-encoder vs late interaction vs cross-encoder

MaxSim scoring

Practical notes

Related