Promptfoopromptfoo 0.124.1 · Node 24 · Python 3
Dashboard
0%

Prompts and vars: one prompt, many questions

Vars are the values a prompt template leaves blank: one prompt with a {{question}} hole runs once per test, each test filling the hole with its own value.

Last updated: 15 Oct, 2026 · Promptfoo 0.124.1

The A Python provider: your own app under test lesson ran two questions through one prompt. That is the shape of almost every suite: a single prompt template and a table of inputs. This lesson writes the template once and lets a list of tests drive many questions through it, then shows what happens when a var name does not line up.

A template with one hole

A prompt is a string with {{name}} placeholders. Here the template wraps each question in a sentence, and {{question}} is the hole every test fills.

yaml
prompts:
  - "A customer asks: {{question}}"

A test per question

Each entry under tests carries a vars block naming the value for this run and an assert for the answer. The template does not change; only the var does.

yaml
  - vars:
      question: Where is order A17?
    assert:
      - type: contains
        value: 3 March

Running three questions through one prompt

Three tests, one template. Promptfoo renders the prompt once per test, calls the provider, and checks each answer.

yaml
providers:
  - file://support_bot.py
prompts:
  - "A customer asks: {{question}}"
tests:
  - vars:
      question: Where is order A17?
    assert:
      - type: contains
        value: 3 March
  - vars:
      question: How long does a refund take?
    assert:
      - type: contains
        value: five working days
  - vars:
      question: Where is order B99?
    assert:
      - type: contains
        value: could not find
Project files used on this pageThis lesson builds on a project from earlier lessons. The code below imports this file. Click a file to see its code, or follow the link to the lesson that wrote it. To run the code yourself, keep it in the same folder.
View the code here
support_bot.py
# the shop support bot: find the answer in the handbook, or say it is missing
HANDBOOK = {
    "a17": "Order A17 shipped on 3 March by courier.",
    "refund": "Refunds take five working days.",
}

def call_api(prompt, options, context):
    asked = prompt.lower()                      # the rendered prompt is the question
    for key, line in HANDBOOK.items():
        if key in asked:
            return {"output": line}
    return {"output": "I could not find that in the handbook."}
Exampleone prompt template, three questions, each with its own var
promptfoo eval --no-table

A misspelled var renders an empty hole

The var name in a test must match the name in the template. If it does not, promptfoo does not error: it fills the hole with nothing and runs on. Here the test spells the var questoin, so {{question}} has no value and renders empty. The box table would still show the template {{question}} in its header, so the hole is invisible there. Writing the run to a file and reading results.results[].prompt.raw shows the prompt that ran.

yaml
providers:
  - file://support_bot.py
prompts:
  - "A customer asks: {{question}}"
tests:
  - vars:
      questoin: Where is order A17?
Examplethe typo leaves a hole in the prompt, with no error
promptfoo eval -o out.json >/dev/null 2>&1

# read back the prompt that ran, and the reply
python3 - <<'PY'
import json
r = json.load(open("out.json"))["results"]["results"][0]
print("vars       :", r["vars"])
print("prompt.raw :", repr(r["prompt"]["raw"]))
print("output     :", repr(r["response"]["output"]))
PY

What the hole shows

  • The prompt rendered to 'A customer asks: ', with nothing where the question should be. The misspelled var was never used.
  • No error was raised. A missing or misspelled var is a silent empty string, not a failure, so a typo can pass unnoticed.
  • The bot answered its fallback, because an empty question matches no handbook word. The hole changed the answer without any warning.

Where vars earn their keep

  • A table of questions run through one prompt, so adding a case is adding a row.
  • Several vars in one template, such as a question and a customer name filled together.
  • Sharing a prompt across suites, where only the vars differ from one file to the next.
Watch out. A misspelled var does not stop the run; it renders empty. If an answer looks wrong for no reason, check that every {{name}} in the template has a matching var, and read prompt.raw from the JSON output rather than trusting the table header, which shows the template.
Try it yourself
  • Fix the typo back to question and confirm the prompt fills and the answer returns.
  • Add a second var, such as a customer name, to the template and to one test.
  • Read results.results[].prompt.raw for a correct test and compare it with the template in the table header.

You understood something today that you didn't yesterday.