Reading the result table
The result table is the grid promptfoo prints after a run: one row per test, one column per provider, each cell marked [PASS] or [FAIL] next to the answer.
Last updated: 15 Oct, 2026 · Promptfoo 0.124.1
The first-config lesson ran with --no-table for a clean summary. Drop that flag and promptfoo draws the full grid, which is where you see which answer failed and what the bot said in reply. To read a failure, the suite needs one.
Adding a test the bot fails
The promptfooconfig.yaml here keeps the two passing tests and adds a third whose expectation the bot cannot meet: it asks about an order the handbook does not know and expects the word shipped.
- vars:
question: Where is order B99?
assert:
- type: contains
value: shipped
The bot has no entry for order B99, so it returns its fallback line, which does not contain shipped. That test will fail, and the grid will show why.
View the code here
# the shop support bot: find the answer in the handbook, or say it is missing
HANDBOOK = {
"a17": "Order A17 shipped on 3 March by courier.",
"refund": "Refunds take five working days.",
}
def call_api(prompt, options, context):
asked = prompt.lower() # the rendered prompt is the question
for key, line in HANDBOOK.items():
if key in asked:
return {"output": line}
return {"output": "I could not find that in the handbook."}
providers:
- file://support_bot.py
prompts:
- "{{question}}"
tests:
- vars:
question: Where is order A17?
assert:
- type: contains
value: 3 March
- vars:
question: How long does a refund take?
assert:
- type: contains
value: five working days
- vars:
question: Where is order B99?
assert:
- type: contains
value: shipped
Running with the table
Running promptfoo eval without --no-table prints the grid. Because one test fails, the command exits with code 100 rather than 0, which is why this block is marked as one that is expected to exit non-zero.
promptfoo evalStarting evaluation eval-2j4-2026-10-14T19:03:36
Running 3 test cases (up to 4 at a time)...
Creating cache folder at ~/.promptfoo/cache.
┌────────────────────────────────────────────────────────────┬────────────────────────────────────────────────────────────┐
│ question │ [python:support_bot.py:default] {{question}} │
├────────────────────────────────────────────────────────────┼────────────────────────────────────────────────────────────┤
│ Where is order A17? │ [PASS] Order A17 shipped on 3 March by courier. │
├────────────────────────────────────────────────────────────┼────────────────────────────────────────────────────────────┤
│ How long does a refund take? │ [PASS] Refunds take five working days. │
├────────────────────────────────────────────────────────────┼────────────────────────────────────────────────────────────┤
│ Where is order B99? │ [FAIL] I could not find that in the handbook. │
└────────────────────────────────────────────────────────────┴────────────────────────────────────────────────────────────┘
✓ Eval complete (ID: eval-2j4-2026-10-14T19:03:36)
» View results: promptfoo view
» Share with your team: https://promptfoo.app
» Feedback: https://promptfoo.dev/feedback
Results:
✓ 2 passed (66.67%)
✗ 1 failed (33.33%)
0 errors (0%)
Duration: 0s (concurrency: 4)Reading the cells
- The left column is the var, the
questioneach row was run with. - The right column is the provider; its data cells start with
[PASS]or[FAIL]and then the bot's answer. - Two rows pass and one fails, and the summary underneath reports 66.67% passed and 33.33% failed.
- The failing row shows the fallback line, the answer that did not contain
shipped.
The header shows the template, not the value
The provider column's header reads [python:support_bot.py:default] {{question}}. That {{question}} is the raw prompt template, not the question that ran; only the data cells below it show the rendered value. If a var were mistyped the prompt would render empty and the header would still show the template, so it cannot tell you the prompt that ran. To read the exact rendered prompt, write the run to a file with -o out.json and read results.results[].prompt.raw there.
The table vs the summary
| The summary (--no-table) | The full table | |
|---|---|---|
| What it shows | Counts and the pass rate | Every answer, cell by cell |
| Best for | A quick check or a build | Seeing why a test failed |
| Width | A few short lines | A fixed wide grid |
Where the table helps
- Debugging a failure, where you need the exact answer the bot gave.
- Comparing providers later, when each gets its own column side by side.
- A small suite you read by eye, before it grows too wide for the terminal.
promptfoo eval exit with code 100, not 0. In a terminal that is easy to miss, but a build job reads the exit code and will fail on it. The failing-a-build lesson turns that exit code into a build gate.Related
- Previous: promptfoo eval with a config file
- Next: promptfoo view: every run kept
- Reference: Promptfoo outputs docs
- Change the third test's expected value to
could not findand rerun; watch all three pass. - Run the same config with
-o out.jsonand open the file to findresults.results[0].prompt.raw. - Run
promptfoo eval; echo $?and read the exit code after a failing run.
This is what real progress feels like.