Embeddings and a vector store

Search needs each text as a list of numbers, its embedding. A vector store keeps chunks with their embeddings and returns the closest ones to a question.

An embedding model turns text into a vector so that similar texts get similar vectors. Hosted ones are trained to capture meaning. This one counts words, which is enough to see how the rest works, and it runs on your machine.

import re
import zlib

from langchain_core.embeddings import Embeddings

COMMON = {"a", "an", "and", "are", "can", "do", "does", "for", "how", "i",
          "if", "is", "it", "my", "of", "on", "the", "to", "what", "with", "you", "your"}


class WordEmbeddings(Embeddings):
    def embed_query(self, text):
        vector = [0.0] * 256
        for word in re.findall(r"[a-z]+", text.lower()):
            if word not in COMMON:
                vector[zlib.crc32(word.rstrip("s").encode()) % 256] += 1.0
        return vector

    def embed_documents(self, texts):
        return [self.embed_query(text) for text in texts]

Each word is hashed into one of 256 slots and counted. Common words like the and you are skipped, because they appear everywhere and would make every text look alike. Stripping a final s makes "refund" and "refunds" count as one word. Embeddings asks for two methods: one for a question and one for a list of documents.

model = WordEmbeddings()
for text in ["refund", "Refunds", "shipping"]:
    vector = model.embed_query(text)
    print(f"{text:<9} slots {[i for i, v in enumerate(vector) if v]}")

"refund" and "Refunds" land in the same slot, and "shipping" lands somewhere else. Texts about the same thing fill the same slots, which is what a search has to work with.

A vector store

from langchain_core.vectorstores import InMemoryVectorStore
from policies import chunks
from word_embeddings import WordEmbeddings

store = InMemoryVectorStore(WordEmbeddings())
store.add_documents(chunks)

InMemoryVectorStore embeds each chunk as it is added and keeps it in a list. policies.py is lesson 26's file.

for question in ["Is shipping free?", "How do I reset my password?"]:
    doc, score = store.similarity_search_with_score(question, k=1)[0]
    print(f"{score:.2f} {doc.metadata['source']:<12} {question}")

The score is the cosine similarity of the two vectors: 1 for the same words in the same proportions, 0 for nothing in common. Both questions found the policy that answers them.

Where counting words fails

for question in ["Can I pay with bitcoin?", "Do you sell gift cards?"]:
    doc, score = store.similarity_search_with_score(question, k=1)[0]
    print(f"{score:.2f} {doc.metadata['source']:<12} {doc.page_content[:40]}")

Neither question is covered, but the store always returns its closest chunk. Bitcoin landed on the express shipping chunk because two different words fell into the same slot, a collision that hashing cannot avoid. Low scores are the signal: lesson 28 treats anything under 0.3 as no answer.

A retriever

retriever = store.as_retriever(search_kwargs={"k": 1})
print(retriever.invoke("Is shipping free?")[0].page_content)

as_retriever wraps the store as a retriever: an object that takes a question and returns documents, with the same invoke and batch methods as a chat model. There is also a similarity_score_threshold search type for filtering by score; with InMemoryVectorStore in this version it raises NotImplementedError, so lesson 28 filters by score itself.