Output files: json, csv and junit
The -o flag writes a run to a file, and the extension you give it picks the format: out.json for a full record, out.csv for a spreadsheet, out.junit.xml for a build server.
Last updated: 15 Oct, 2026 · Promptfoo 0.124.1
Every run so far was read off the terminal. To feed a run to another tool, a build server or a spreadsheet, promptfoo writes it to a file instead. The Reading the result table lesson read a run by eye; this one writes the same run to three formats and reads a field back out of the JSON.
One run, three files
-o takes a filename and can be repeated, so a single run writes as many files as you ask for. Promptfoo looks at each extension to decide the format, which is why the name, not a flag, is what chooses JSON over CSV over JUnit.
Syntax: promptfoo eval -o <file> -o <file> ..., one -o per output file.
The suite is the shop support bot with one answer it gets right and one it cannot, so the files hold a pass and a fail to look at. Running it writes all three at once, prints the JUnit file, then reads the timestamp and each verdict out of the JSON with a few lines of Python. The bot is the same one from the earlier parts, kept here so the run has a provider to call.
HANDBOOK = {
"a17": "Order A17 shipped on 3 March by courier.",
"refund": "Refunds take five working days.",
}
def call_api(prompt, options, context):
asked = prompt.lower() # promptfoo passes the rendered prompt here
for key, line in HANDBOOK.items():
if key in asked:
return {"output": line}
return {"output": "I could not find that in the handbook."}providers:
- file://support_bot.py
prompts:
- "{{question}}"
tests:
- vars:
question: Where is order A17?
assert:
- type: contains
value: 3 March
- vars:
question: Where is order B99?
assert:
- type: contains
value: shipped
# write the one run to three files at once; the extension picks each format
promptfoo eval --no-table -o out.json -o out.csv -o out.junit.xml
# the JUnit file: it only comes out as real JUnit with a .junit.xml name
cat out.junit.xml
# read fields straight from the JSON: when it ran and each test's verdict
python3 - <<'PY'
import json
d = json.load(open("out.json"))
print("run at:", d["results"]["timestamp"])
for r in d["results"]["results"]:
print(r["success"], "-", r["response"]["output"])
PYStarting evaluation eval-CGH-2026-10-14T19:19:52
Running 2 test cases (up to 4 at a time)...
Creating cache folder at ~/.promptfoo/cache.
✓ Eval complete (ID: eval-CGH-2026-10-14T19:19:52)
» View results: promptfoo view
» Share with your team: https://promptfoo.app
» Feedback: https://promptfoo.dev/feedback
Results:
✓ 1 passed (50.00%)
✗ 1 failed (50.00%)
0 errors (0%)
Duration: 0s (concurrency: 4)
Writing output to out.json, out.csv, out.junit.xml
<?xml version="1.0" encoding="UTF-8"?>
<testsuites errors="0" failures="1" name="promptfoo" skipped="0" tests="2" time="0.397">
<testsuite errors="0" failures="1" name="[python:support_bot.py:default] prompt 1" skipped="0" tests="2" time="0.397" timestamp="2026-10-14T19:19:52.105Z">
<testcase classname="[python:support_bot.py:default] prompt 1" name="test 1" time="0.199"></testcase>
<testcase classname="[python:support_bot.py:default] prompt 1" name="test 2" time="0.198">
<failure message="Assertion failed" type="assertion">Score: 0
Reason: Assertion failed
Failed assertions:
- contains</failure>
</testcase>
</testsuite>
</testsuites>
run at: 2026-10-14T19:19:52.105Z
True - Order A17 shipped on 3 March by courier.
False - I could not find that in the handbook.What each format is for
| File | What it holds | Who reads it |
|---|---|---|
out.json | The whole run: every result, score, assertion and the run stats | Your own code, by field |
out.csv | One row per test with the status, score and grader reason | A spreadsheet or a reviewer |
out.junit.xml | A JUnit report, one <testcase> per test with a <failure> on each fail | A build server's test view |
Reading a field out of the JSON
The JSON nests the per-test records under results.results. Each record carries a success boolean, the answer under response.output, the test's vars, and a gradingResult.componentResults list with one entry per assertion. The read above pulled success and the output; a build could read the same fields to post a summary or fail on a named test.
- The timestamp sits under
results.timestampas an ISO string, the moment the run started. - Each
results.results[i].successis the verdict for one test, hereTruefor the A17 answer andFalsefor the one the bot missed. - The run stats live under
results.stats, with the pass, fail and error counts and the token usage.
Why the JUnit name matters
Writing -o out.xml gives you promptfoo's own XML, not JUnit. The JUnit format, the one a build server reads as a test report, only comes out when the filename ends in .junit.xml. The failing test above shows as a <failure> element inside its <testcase>, which is what a build server renders as a red test.
-o out.xml and -o out.junit.xml are not the same file with a longer name. Plain .xml is promptfoo-native XML that a JUnit reader will not understand; only the .junit.xml ending produces a JUnit report. If your build server shows no tests, check the extension first.Where output files help
- A build server's test tab, fed the
.junit.xmlso each eval case shows as a passing or failing test. - A dashboard or a diff, reading
out.jsonfield by field to track scores across runs. - A reviewer who lives in a spreadsheet, handed the CSV with one row per question.
Related
- Previous: Filtering runs: re-running the failures
- Next: Failing a build: exit codes and GitHub Actions
- Reference: Promptfoo documentation
- Add
-o out.yamlto the command and open the YAML form of the same run. - Change the Python read to print
r["gradingResult"]["componentResults"][0]["reason"]for each test. - Write
-o out.xmlinstead and compare it without.junit.xml; note which one a JUnit reader would accept.
Every expert started right here.