BM25 retrieval: keyword search

InMemoryBM25Retriever ranks documents by BM25, a keyword score that rewards words the query shares with a document, especially rare ones.

for document in retriever.run(query="how long do refunds take")["documents"]:
    print(f"{document.score:.3f}", document.content)

Both documents contain refunds, the only query word that appears in the store, once each, and they are about the same length, so they scored the same. top_k=2 returned the best two. Between documents with the same matches, BM25 prefers the shorter one, since the word is a bigger part of it. Scores are only comparable within one query.

print(retriever.run(query="money back")["documents"])

"Money back" shares no words with any document, so nothing matches. That is BM25's weak spot, and why lesson 9 adds embeddings. Its strengths are that it needs no model, is fast, and matches exact terms like order numbers and product codes that embeddings blur.

Settings at run time

print(len(retriever.run(query="refunds", top_k=1)["documents"]))
print([round(float(d.score), 3) for d in retriever.run(query="refunds", scale_score=True)["documents"]])

top_k and scale_score can be passed on each run. scale_score=True maps scores into 0 to 1, which makes thresholds easier to choose.