Embedding retrieval: search by meaning

Embedding retrieval compares vectors instead of words. A document embedder adds a vector to each document; a text embedder turns the query into one.

Real embedders run a model: SentenceTransformersDocumentEmbedder downloads one, OpenAIDocumentEmbedder needs a key. These two components use the same inputs and outputs with a word-hashing function instead, so the pipeline code is what a real one looks like:

Exampleword_embedders.py

import re

from haystack import Document, component


def vector(text):
    """A 32-number vector from the words of a text. Same words, similar vectors."""
    slots = [0.0] * 32
    for word in re.findall(r"[a-z]+", text.lower()):
        if len(word) > 3:
            slots[sum(map(ord, word.rstrip("s"))) % 32] += 1.0
    length = sum(x * x for x in slots) ** 0.5 or 1.0
    return [x / length for x in slots]


@component
class WordDocumentEmbedder:
    @component.output_types(documents=list[Document])
    def run(self, documents: list[Document]):
        return {"documents": [Document(content=d.content, meta=d.meta, embedding=vector(d.content)) for d in documents]}


@component
class WordTextEmbedder:
    @component.output_types(embedding=list[float])
    def run(self, text: str):
        return {"embedding": vector(text)}

Example

store = InMemoryDocumentStore(embedding_similarity_function="cosine")
documents = [
    Document(content="Refunds are paid within five working days."),
    Document(content="Parcels ship within two days and arrive by courier."),
]
store.write_documents(WordDocumentEmbedder().run(documents)["documents"])

search = Pipeline()
search.add_component("embed", WordTextEmbedder())
search.add_component("retrieve", InMemoryEmbeddingRetriever(document_store=store, top_k=2))
search.connect("embed.embedding", "retrieve.query_embedding")

for document in search.run({"embed": {"text": "when is my refund paid"}})["retrieve"]["documents"]:
    print(f"{document.score:.3f}", document.content)

The document embedder filled each document's embedding; the store keeps it. At query time the text embedder's vector goes to InMemoryEmbeddingRetriever, which ranks by cosine similarity. "refund" and "refunds" hash to the same slot here because the function strips a final s; a trained model also knows that "money back" is close, which is the reason to use one.

A trained embedding model

Example

from haystack.components.embedders import SentenceTransformersDocumentEmbedder, SentenceTransformersTextEmbedder

document_embedder = SentenceTransformersDocumentEmbedder(model="sentence-transformers/all-MiniLM-L6-v2")
text_embedder = SentenceTransformersTextEmbedder(model="sentence-transformers/all-MiniLM-L6-v2")

Same connections, with pip install sentence-transformers, which downloads the model on first use. Always embed documents and queries with the same model. Many projects combine BM25 and embeddings, joining both result lists, to get exact terms and meaning.

Try it yourself

Query "courier" and read the scores.
Write a document without an embedding into the store and search again.
Set embedding_similarity_function="dot_product".

PreviousMetadata filters: who may see what

This is what real progress feels like.