One model call, or three

Lesson 10 ended with a message that matched no flow. This lesson counts what that costs, because the gap between a matched message and an unmatched one is one model call against three, and it explains most of the surprises still to come.

Matched, unmatched, and nothing defined

import pretend_nemo
from nemoguardrails import LLMRails, RailsConfig

rails = LLMRails(RailsConfig.from_path("."))
for ask in ["When do I get my money back?", "Is the shop open on Sunday?"]:
    rails.generate(messages=[{"role": "user", "content": ask}])
    print(ask, [call.task for call in rails.explain().llm_calls])

A matched message costs one call. The runtime works out the canonical form and stops asking, because the flow says what the bot does next and define bot already holds the words.

An unmatched message costs three. After generate_user_intent comes generate_next_steps, which asks the model what the bot should do, and then generate_bot_message, which asks for the actual sentence.

import pretend_nemo
from nemoguardrails import LLMRails, RailsConfig

rails = LLMRails(RailsConfig.from_path("."))
rails.generate(messages=[{"role": "user", "content": "When do I get my money back?"}])
print([call.task for call in rails.explain().llm_calls])

A configuration with no define user blocks costs one call, named general. With nothing to recognise there are no dialog rails at all, so the runtime hands the question over and hands the answer back. That folder has a config.yml and no rails.co.

The three steps, drawn

Each step is skipped when the configuration already answers it. A flow answers step two. A define bot block answers step three.

A trap worth knowing

If step two returns a bot intent that some define bot block already defines, step three never runs and the canned sentence comes back instead. NeMo ships several of these of its own, including inform answer unknown, so a model that happens to answer with that name sounds like it gave up when it was about to answer properly.