Writing the stand-in embedder

The model decides what to keep. The embedder is what makes it findable, and it is the piece that makes search feel like magic when it works.

An embedder turns text into a list of numbers. The only rule that matters is that texts meaning similar things should produce similar lists, because search works by measuring how close two lists are.

class PretendEmbedder(Embeddings):
    """One vector per text, built from its words. Same words, same vector."""

    def embed_query(self, text):
        vector = [0.0] * 64
        for word in WORD.findall((text or "").lower()):
            vector[sum(ord(c) for c in word) % 64] += 1.0
        size = sum(v * v for v in vector) ** 0.5 or 1.0
        return [v / size for v in vector]

    def embed_documents(self, texts):
        return [self.embed_query(t) for t in texts]

Sixty-four numbers, one slot per bucket. Every word in the text adds one to the slot its letters add up to, and the whole list is then scaled to length one so that a long memory does not beat a short one just for being long.

Two texts sharing words land in the same slots, so they end up close. Two texts sharing meaning but not words do not, which is exactly the limitation to keep in mind.

Watching it work

from pretend_mem0 import PretendEmbedder

embed = PretendEmbedder()
a = embed.embed_query("I prefer email updates")
b = embed.embed_query("email updates please")
c = embed.embed_query("deliver to the office")

near = lambda x, y: round(sum(i * j for i, j in zip(x, y)), 2)
print("shared words ", near(a, b))
print("no overlap   ", near(a, c))

Nearly one for the pair that shares words and nothing for the pair that does not. That single number is what ranks every search result in this course.

Sixty-four, and why the number matters

The store has to be told the same length. Say 64 here and 1536 in the store configuration and Mem0 will fail on the first insert, complaining about dimensions. It is the most common setup error when swapping an embedder, including the swap to a real one in lesson 19.