Gemini embeddings in place of word counts

The stand-in matches spelling, not meaning, and sometimes not even spelling. A real embedding model fixes that, and with unstructured 0.27.6 the dependable way is a small class of your own.

Where the stand-in goes wrong

embedder = ShopEmbedder()
embedder.embed_documents(chunks)
question = embedder.embed_query("how long does express delivery take")
scored = sorted(((cosine(question, c.embeddings), c) for c in chunks),
                key=lambda pair: pair[0], reverse=True)
for score, chunk in scored[:2]:
    print(round(score, 2), chunk.metadata.filename, "|", chunk.text[:30])
print(zlib.crc32(b"ravi") % SLOTS, zlib.crc32(b"delivery") % SLOTS)

The best match for a delivery question is a chunk containing the single word Ravi, the name at the end of the email. ravi and delivery land in the same one of the 64 slots, so to the stand-in they are the same word. The shipping chunk that answers the question comes second.

A real embedding model has no slots to share. It is trained so that text with similar meaning gets similar numbers, which lets a question match an answer that uses different words.

The encoders unstructured still ships

from unstructured.embed.huggingface import (
    HuggingFaceEmbeddingConfig,
    HuggingFaceEmbeddingEncoder,
)

try:
    HuggingFaceEmbeddingEncoder(config=HuggingFaceEmbeddingConfig())
except TypeError as error:
    print(error)

The library has an unstructured.embed package with encoders for OpenAI, Hugging Face, Amazon Bedrock, Vertex AI, Voyage AI, Mixedbread and OctoAI. In version 0.27.6 the Hugging Face and Bedrock encoders cannot even be created, because they never implement initialize. The OpenAI encoder can be, but it defaults to text-embedding-ada-002, goes through LangChain, and has no setting for another provider's URL.

Unstructured's own documentation now says the open source library has no built-in support for generating embeddings. It points to embedding as a separate step, or to the Unstructured Ingest tools. Writing the encoder yourself follows that advice and keeps the BaseEmbeddingEncoder shape the handbook already uses.

GeminiEmbedderOptional

Google's Gemini API has a free tier that covers its embedding model, gemini-embedding-2. Create a key at aistudio.google.com/apikey and install Google's SDK.

pip install google-genai

export GOOGLE_API_KEY=AIza...

from dataclasses import dataclass, field

from google import genai
from unstructured.embed.interfaces import BaseEmbeddingEncoder, EmbeddingConfig

MODEL = "gemini-embedding-2"


@dataclass
class GeminiEmbedder(BaseEmbeddingEncoder):
    config: EmbeddingConfig = field(default_factory=EmbeddingConfig)

    def __post_init__(self):
        self.client = genai.Client()  # reads GOOGLE_API_KEY

genai.Client() finds GOOGLE_API_KEY in the environment by itself. The client is made once, when the embedder is created, and reused for every call.

    def initialize(self):
        pass

    @property
    def num_of_dimensions(self):
        return (len(self.embed_query("dimensions")),)

    @property
    def is_unit_vector(self):
        return False

The same three members ShopEmbedder has. Gemini's vectors have a fixed length, so num_of_dimensions embeds a short word and measures the result rather than hard-coding a number that could change with the model.

    def embed_query(self, query):
        result = self.client.models.embed_content(model=MODEL, contents=query)
        return result.embeddings[0].values

    def embed_documents(self, elements):
        result = self.client.models.embed_content(
            model=MODEL, contents=[element.text for element in elements])
        for element, embedding in zip(elements, result.embeddings):
            element.embeddings = embedding.values
        return elements

embed_content accepts a list, so embed_documents sends every chunk in one request instead of one per chunk. A free tier limits requests per minute, and eleven chunks cost one of them.

This code has no captured output on the page, because running it needs your key. Lesson 28's handbook takes the embedder as an argument, so trying Gemini there means changing ShopEmbedder() to GeminiEmbedder() on one line.