The prompt behind the intent

Lesson 9 explained how the closest example utterances are found. This lesson reads the prompt they are pasted into, because that prompt is the whole of what generate_user_intent asks, and reading it is the last piece of the stand-in model.

What the runtime sends

import pretend_nemo
from nemoguardrails import LLMRails, RailsConfig

rails = LLMRails(RailsConfig.from_path("."))
rails.generate(messages=[{"role": "user", "content": "When do I get my money back?"}])
prompt = rails.llm.prompts[0]
start = prompt.index("# This is how the user talks:")
print(prompt[start:prompt.index("# This is the current")].strip())

The example utterances out of rails.co, each with the intent it belongs to, ordered by how close the index judged them to the question. The model's job is to read them and answer with one line beginning User intent:.

Further down the same prompt is a second hint, Choose intent from this list, naming the intents the search turned up. A real model uses both.

Choosing, in six lines

def examples(prompt):
    """The (utterance, intent) pairs the runtime pasted into the prompt."""
    block = prompt.split("# This is how the user talks:")[-1]
    block = block.split("# This is the current conversation")[0]
    return re.findall(r'User message: "(.*)"\nUser intent: (.*)', block)

Pull the examples back out of the prompt. This is the one place the stand-in cheats: a real model reads the whole prompt, and this one reads the part that matters.

def best_intent(prompt):
    """Pick the intent whose example shares the most words with this message."""
    asked = words(last_turn(prompt))
    best, score = "", 0
    for utterance, intent in examples(prompt):
        shared = len(asked & words(utterance))
        if shared > score:
            best, score = intent.strip(), shared
    return best or "ask general question"

Then pick the intent whose example shares the most words with what was actually said. It is a crude judgement and it is a real one: change the examples in rails.co and the answer changes, which is how a real model behaves too.

Seeing the decision

import pretend_nemo
from nemoguardrails import LLMRails, RailsConfig

rails = LLMRails(RailsConfig.from_path("."))
for ask in ["When do I get my money back?", "Is the shop open on Sunday?"]:
    rails.generate(messages=[{"role": "user", "content": ask}])
    print(ask, "->", rails.explain().llm_calls[0].completion.strip())

The first was recognised. The second shares no word with any example, so best_intent fell back to ask general question, which no flow mentions. Lesson 11 follows what happens after that.