Promptfoopromptfoo 0.124.1 · Node 24 · Python 3
Dashboard
0%

Output files: json, csv and junit

The -o flag writes a run to a file, and the extension you give it picks the format: out.json for a full record, out.csv for a spreadsheet, out.junit.xml for a build server.

Last updated: 15 Oct, 2026 · Promptfoo 0.124.1

Every run so far was read off the terminal. To feed a run to another tool, a build server or a spreadsheet, promptfoo writes it to a file instead. The Reading the result table lesson read a run by eye; this one writes the same run to three formats and reads a field back out of the JSON.

One run, three files

-o takes a filename and can be repeated, so a single run writes as many files as you ask for. Promptfoo looks at each extension to decide the format, which is why the name, not a flag, is what chooses JSON over CSV over JUnit.

Syntax: promptfoo eval -o <file> -o <file> ..., one -o per output file.

The suite is the shop support bot with one answer it gets right and one it cannot, so the files hold a pass and a fail to look at. Running it writes all three at once, prints the JUnit file, then reads the timestamp and each verdict out of the JSON with a few lines of Python. The bot is the same one from the earlier parts, kept here so the run has a provider to call.

python
HANDBOOK = {
    "a17": "Order A17 shipped on 3 March by courier.",
    "refund": "Refunds take five working days.",
}

def call_api(prompt, options, context):
    asked = prompt.lower()              # promptfoo passes the rendered prompt here
    for key, line in HANDBOOK.items():
        if key in asked:
            return {"output": line}
    return {"output": "I could not find that in the handbook."}
yaml
providers:
  - file://support_bot.py
prompts:
  - "{{question}}"
tests:
  - vars:
      question: Where is order A17?
    assert:
      - type: contains
        value: 3 March
  - vars:
      question: Where is order B99?
    assert:
      - type: contains
        value: shipped
Exampleone run written to JSON, CSV and JUnit, then read back
# write the one run to three files at once; the extension picks each format
promptfoo eval --no-table -o out.json -o out.csv -o out.junit.xml

# the JUnit file: it only comes out as real JUnit with a .junit.xml name
cat out.junit.xml

# read fields straight from the JSON: when it ran and each test's verdict
python3 - <<'PY'
import json
d = json.load(open("out.json"))
print("run at:", d["results"]["timestamp"])
for r in d["results"]["results"]:
    print(r["success"], "-", r["response"]["output"])
PY

What each format is for

FileWhat it holdsWho reads it
out.jsonThe whole run: every result, score, assertion and the run statsYour own code, by field
out.csvOne row per test with the status, score and grader reasonA spreadsheet or a reviewer
out.junit.xmlA JUnit report, one <testcase> per test with a <failure> on each failA build server's test view

Reading a field out of the JSON

The JSON nests the per-test records under results.results. Each record carries a success boolean, the answer under response.output, the test's vars, and a gradingResult.componentResults list with one entry per assertion. The read above pulled success and the output; a build could read the same fields to post a summary or fail on a named test.

  • The timestamp sits under results.timestamp as an ISO string, the moment the run started.
  • Each results.results[i].success is the verdict for one test, here True for the A17 answer and False for the one the bot missed.
  • The run stats live under results.stats, with the pass, fail and error counts and the token usage.

Why the JUnit name matters

Writing -o out.xml gives you promptfoo's own XML, not JUnit. The JUnit format, the one a build server reads as a test report, only comes out when the filename ends in .junit.xml. The failing test above shows as a <failure> element inside its <testcase>, which is what a build server renders as a red test.

Watch out. -o out.xml and -o out.junit.xml are not the same file with a longer name. Plain .xml is promptfoo-native XML that a JUnit reader will not understand; only the .junit.xml ending produces a JUnit report. If your build server shows no tests, check the extension first.

Where output files help

  • A build server's test tab, fed the .junit.xml so each eval case shows as a passing or failing test.
  • A dashboard or a diff, reading out.json field by field to track scores across runs.
  • A reviewer who lives in a spreadsheet, handed the CSV with one row per question.
Try it yourself
  • Add -o out.yaml to the command and open the YAML form of the same run.
  • Change the Python read to print r["gradingResult"]["componentResults"][0]["reason"] for each test.
  • Write -o out.xml instead and compare it with out.junit.xml; note which one a JUnit reader would accept.

Every expert started right here.