An embedder that needs no API key

Searching chunks means turning each one into a list of numbers. This lesson writes ShopEmbedder, a stand-in that counts words, so the handbook runs with no key and no download.

A real embedding model, the kind lesson 25 plugs in, turns text into hundreds or thousands of numbers that capture meaning. The stand-in does something much simpler that has the same shape: text in, a fixed-length list of numbers out.

Words into numbers

import re
import zlib

SLOTS = 64


def vector(text):
    slots = [0.0] * SLOTS
    for word in re.findall(r"[a-z]{4,}", text.lower()):
        slots[zlib.crc32(word.encode()) % SLOTS] += 1.0
    return slots

counts = vector("Gift cards and opened software are not refundable.")
print(len(counts), sum(counts))
print([slot for slot, count in enumerate(counts) if count])

Every word of four letters or more is hashed into one of 64 slots, and the slot is counted. Five words qualified: gift, cards, opened, software and refundable. Short words such as and and not are skipped.

zlib.crc32 does the hashing rather than Python's hash, because hash gives a string a different value in every new Python process. The same text has to produce the same vector every time, or yesterday's saved vectors would not match today's question.

How close two vectors are

import math


def cosine(a, b):
    dot = sum(x * y for x, y in zip(a, b))
    size = math.sqrt(sum(x * x for x in a)) * math.sqrt(sum(y * y for y in b))
    return dot / size if size else 0.0

question = vector("are gift cards refundable")
print(round(cosine(question, vector("Gift cards and opened software are not refundable.")), 2))
print(round(cosine(question, vector("Refunds go back to the card used for the order.")), 2))

Cosine similarity compares the direction of two vectors: 1.0 when they point the same way, 0.0 when they share nothing. The gift card sentence shares three counted words with the question and scores far higher than the card sentence, which shares none of them exactly. card and cards are different words to a counter.

ShopEmbedder

To be usable wherever unstructured expects an embedder, the stand-in subclasses BaseEmbeddingEncoder. That class asks for five things: initialize, num_of_dimensions, is_unit_vector, embed_documents and embed_query.

@dataclass
class ShopEmbedder(BaseEmbeddingEncoder):
    config: EmbeddingConfig = field(default_factory=EmbeddingConfig)

    def initialize(self):
        pass

    @property
    def num_of_dimensions(self):
        return (SLOTS,)

    @property
    def is_unit_vector(self):
        return False

num_of_dimensions and is_unit_vector describe the vectors to whatever uses the embedder: 64 numbers each, and not scaled to a length of 1. initialize has nothing to set up, because counting needs no model.

    def embed_query(self, query):
        return vector(query)

    def embed_documents(self, elements):
        for element in elements:
            element.embeddings = vector(element.text)
        return elements

embed_documents stores each vector on the element itself, in element.embeddings, so a chunk and its vector travel together. embed_query returns the vector for a question without storing it. The class lives in pretend_unstructured.py with vector, cosine and a best helper that returns the closest chunk and its score.

import make_office
from unstructured.partition.auto import partition
from unstructured.chunking.title import chunk_by_title
from pretend_unstructured import ShopEmbedder, best

chunks = chunk_by_title(partition("refunds.html"), max_characters=200)
ShopEmbedder().embed_documents(chunks)
print(len(chunks), len(chunks[0].embeddings))
score, chunk = best(chunks, "are gift cards refundable")
print(round(score, 2), "|", chunk.text)