Groq or Gemini

Switching providers in NeMo Guardrails means changing the model entry in config.yml: the model name, the base_url and the key variable; the rails stay as they are.

Last updated: 30 Sep, 2026 · NeMo Guardrails 0.24.1

Every lesson ran on Groq's openai/gpt-oss-120b. The video's app lets the user pick from several models. The same freedom is three lines of YAML.

Syntax:

yaml

models:
  - type: main
    engine: openai
    model: <the provider's model name>
    api_key_env_var: <the key's variable>
    parameters:
      base_url: <the provider's OpenAI-compatible URL>

A second Groq model

openai/gpt-oss-20b is smaller and faster, on the same free key.

yaml

models:
  - type: main
    engine: openai
    model: openai/gpt-oss-20b
    api_key_env_var: GROQ_API_KEY
    parameters:
      base_url: https://api.groq.com/openai/v1
      temperature: 0

Project files used on this pageThis lesson builds on a project from earlier lessons. The code below imports this file. Click a file to see its code, or follow the link to the lesson that wrote it. To run the code yourself, keep it in the same folder.

View the code here

config.yml

models:
  - type: main
    engine: openai
    model: openai/gpt-oss-20b
    api_key_env_var: GROQ_API_KEY
    parameters:
      base_url: https://api.groq.com/openai/v1
      temperature: 0

instructions:
  - type: general
    content: |
      You are an Enterprise IT Assistant specialising in Kubernetes,
      Intel hardware, and enterprise networking.
      Only answer questions about these topics.
      Answer in one or two short sentences.

The same question on gpt-oss-20b

ExampleAPI key

from nemoguardrails import LLMRails, RailsConfig

rails = LLMRails(RailsConfig.from_path("."))


def chat(message):
    reply = rails.generate(messages=[{"role": "user", "content": message}])
    print("User:", message)
    print("Bot :", reply["content"])

chat("What is BGP?")

Output

User: What is BGP?
Bot : BGP is the Border Gateway Protocol, the primary inter‑domain routing protocol that exchanges reachability information between autonomous systems on the Internet.

Gemini and OpenRouter

Provider	model	base_url	Key variable
Groq	`openai/gpt-oss-120b`	`https://api.groq.com/openai/v1`	`GROQ_API_KEY`
Gemini	`gemini-2.5-flash`	`https://generativelanguage.googleapis.com/v1beta/openai/`	`GEMINI_API_KEY`
OpenRouter	a model ending in `:free`	`https://openrouter.ai/api/v1`	`OPENROUTER_API_KEY`

What changes with the model

The prompts. NeMo picks its prompt set by the model's name, as The prompt behind the intent showed. Check colang_history after a switch; keep prompts.yml if the new model answers the user instead of naming the intent.
The self-check answers. A different model may phrase Yes and No differently; the parser reads only the first two words.
The embedding search is separate from the chat model and does not change.

Same provider vs another provider

Change	Lines to edit
Another Groq model	`model`
Another OpenAI-compatible provider	`model`, `base_url`, `api_key_env_var`
A provider with its own API	A LangChain package and `NEMOGUARDRAILS_LLM_FRAMEWORK=langchain`

When to switch

The free quota of one provider is used up.
A rail needs a stronger model: the video says its jailbreak layer failed until it used a better reasoning model.

Watch out. The self-check rails use the main model unless told otherwise. Moving the main model to a small, fast one also moves every input and output check to it.

Try it yourself

Run the topic guard from define flow on gpt-oss-20b and read the history.
Add a second model entry with type: self_check_input pointing at gpt-oss-20b.

PreviousTracing with Logfire

You understood something today that you didn't yesterday.

Groq or Gemini

A second Groq model

The same question on gpt-oss-20b

Gemini and OpenRouter

What changes with the model

Same provider vs another provider

When to switch

Related