BM25: when exact words matter

BM25 is a keyword ranking that scores a chunk by how many of the question's words it contains, weighting rare words most, with no embedding model.

Last updated: 28 Sep, 2026 · LlamaIndex 0.14

Part 3 answered questions by meaning. But a part number carries almost no meaning to embed. For codes, names and exact phrases, keyword ranking finds what semantic search misses, so this part combines the two.

Building a keyword retriever beside the semantic one

The BM25 integration builds its index from the same nodes as the vector index. Both retrievers take a query and return scored chunks, so they are easy to compare side by side.

python

from llama_index.retrievers.bm25 import BM25Retriever

keyword = BM25Retriever.from_defaults(nodes=nodes, similarity_top_k=2)
semantic = index.as_retriever(similarity_top_k=2)

Project files used on this pageThis lesson builds on a project from earlier lessons. The code below imports these files. Click a file to see its code, or follow the link to the lesson that wrote it. To run the code yourself, keep them in the same folder.

View the code here

help/lamps.md

# Lamps

The LMP-204 desk lamp has a known cable fault. Stop using a lamp with a damaged cable and we will replace it free of charge.

All lamps come with a two year guarantee against electrical faults.

Bulbs are not covered by the refund policy once they have been used.

The LMP-310 floor lamp needs a bulb with an E27 fitting, which is sold separately.

help/refunds.md

# Refunds

You can get a full refund within 30 days of delivery. The money goes back to the card you paid with within 5 working days of us receiving the item.

Items bought in a sale can be refunded too, but the delivery charge is not returned.

To start a refund, open the order in your account and choose Return an item. Print the label and drop the parcel at any post office.

Personalised items cannot be refunded unless they arrive damaged.

help/delivery.md

# Delivery

Standard delivery takes 3 to 5 working days and is free on orders over 40.

Express delivery arrives the next working day if you order before 2pm. It costs 6.

We deliver to the mainland only. Parcels to islands take 2 extra working days.

If a parcel has not arrived after 10 working days, contact us and we will send a replacement.

Comparing keyword and semantic on three questions

Example

from llama_index.retrievers.bm25 import BM25Retriever

keyword = BM25Retriever.from_defaults(nodes=nodes, similarity_top_k=2)
semantic = index.as_retriever(similarity_top_k=2)
for question in ["LMP-204", "cable fault on my lamp", "Is reimbursement possible?"]:
    print(question)
    print("   keyword: ", [(n.metadata["file_name"], round(n.score, 2)) for n in keyword.retrieve(question)])
    print("   semantic:", [(n.metadata["file_name"], round(n.score, 2)) for n in semantic.retrieve(question)])

Output

LMP-204
   keyword:  [('lamps.md', 0.85), ('lamps.md', 0.54)]
   semantic: [('lamps.md', 0.35), ('lamps.md', 0.21)]
cable fault on my lamp
   keyword:  [('lamps.md', 2.21), ('lamps.md', 0.54)]
   semantic: [('lamps.md', 0.61), ('lamps.md', 0.28)]
Is reimbursement possible?
   keyword:  [('refunds.md', 0.0), ('delivery.md', 0.0)]
   semantic: [('refunds.md', 0.33), ('refunds.md', 0.3)]

Reading the keyword and semantic scores

For LMP-204, keyword wins cleanly. It scores the chunk naming that exact part highest; semantic ranks the same way but with weaker scores, since a part number is near-meaningless to embed.
For a reworded refund question, keyword scores zero. "Is reimbursement possible?" shares no word with any chunk, so every keyword score is 0 and the order is meaningless, while semantic finds both refund chunks.
Each method fails where the other succeeds. That is the case for combining them next.

Keyword vs semantic search

	BM25 keyword	Semantic
Matches	Shared words, rare ones most	Closeness of meaning
Best at	Codes, names, exact phrases	Reworded questions, synonyms
Blind to	Synonyms and paraphrase	Strings with little meaning

When to reach for BM25

Customers search by part numbers, error codes, policy numbers or legal clause references.
Exact names and phrases matter more than meaning, like a product SKU.
You want a keyword baseline to compare a semantic retriever against.

Watch out

A BM25 score of 0 does not mean the last place; it means no query word appeared, so the ranking among zeros is arbitrary. Do not read order into all-zero results, and pair BM25 with semantic search in Hybrid search: keywords and meaning together.

Previous: Chat engines: remembering the conversation
Next: Hybrid search: keywords and meaning together

Try it yourself

Search both ways for "2pm".
Search both ways for "parcel late".
Build the BM25 retriever with similarity_top_k=4 and look for zero scores.

PreviousChat engines: remembering the conversation

Little by little, you're building something great.