The embedding model, and the download it replaces

The config.yml in lessons 7 and 8 carried a second model with type: embeddings, with a promise that this lesson would explain it. It is the piece that turns "when do I get my money back" into ask refund policy without anybody writing a pattern.

What NeMo does if you say nothing

Leave the embeddings model out and the configuration still works. On a machine that has never run NeMo Guardrails before, the first question then takes a surprisingly long time: the runtime defaults to the engine FastEmbed and the model all-MiniLM-L6-v2, and fetches about ninety megabytes before it can answer.

That is a sensible default in production and a poor one in a tutorial. Registering your own embedding model removes it, and the whole class is fourteen lines.

The class

import zlib

from nemoguardrails.embeddings.providers import register_embedding_provider
from nemoguardrails.embeddings.providers.base import EmbeddingModel


class PretendEmbeddings(EmbeddingModel):
    engine_name = "pretend"
    WIDTH = 64

    def __init__(self, embedding_model="pretend-embed", **kwargs):
        self.model_name = embedding_model
        self.embedding_size = self.WIDTH

EmbeddingModel is an abstract base class with two methods to fill in. engine_name is the name a configuration will ask for, and embedding_size is how long the vectors are.

    def encode(self, documents):
        return [self.vector(d) for d in documents]

    async def encode_async(self, documents):
        return self.encode(documents)

    def vector(self, text):
        out = [0.0] * self.WIDTH
        for word in words(text):
            out[zlib.crc32(word.encode()) % self.WIDTH] = 1.0
        size = sum(out) ** 0.5 or 1.0
        return [v / size for v in out]

The rest of the class. vector gives every word its own slot, chosen by a hash, then scales the result to length one. Two sentences that share words point in a similar direction; two that share nothing sit at right angles.

Seeing it work

from pretend_nemo import PretendEmbeddings

embed = PretendEmbeddings()
a, b, c = embed.encode(["Where is my order?", "Has order A17 shipped?", "Rain fell all week."])
print(round(sum(x * y for x, y in zip(a, b)), 3))
print(round(sum(x * y for x, y in zip(a, c)), 3))

The first pair share the word order and score above zero. The second pair share nothing and score exactly zero. That is the whole mechanism: the runtime embeds the incoming message, compares it against every example utterance in every define user block, and keeps the closest few.

Registering it, once

from nemoguardrails.embeddings.providers import register_embedding_provider

try:
    register_embedding_provider(PretendEmbeddings, "pretend")
except ValueError as error:
    print("ValueError:", error)

Importing pretend_nemo already registered it, and the embedding registry refuses a second entry under the same name. Worth knowing, because the chat provider registry behaves the opposite way: register_provider is a plain dictionary write and calling it twice is harmless.

Subclassing EmbeddingModel and setting engine_name registers nothing on its own. The call is what does it.