Promptfoopromptfoo 0.123.0 · Node 22.22+ · Python 3
0%
1
Curious builder0 XP earned · 300 to level 2
0 daysFinish a lesson to begin
Badge collection0 of 6 unlocked
25 small wins to finish your pathNext lesson

Output files: json, csv and junit

The table is for you. Everything else that wants these results, a dashboard, a spreadsheet, a build server, wants a file.

-o takes a path and picks the format from the extension. The same run can write several.

bash
promptfoo eval -o results.json -o results.csv -o results.junit.xml

What is in the JSON

The JSON is the complete record: the config that ran, every answer, every assertion with its score and reason, and a summary. Two parts of it are worth knowing by name.

Example
promptfoo eval -o results.json > /dev/null
python3 -c "
import json
d = json.load(open('results.json'))
print(json.dumps(d['results']['stats']['successes']), 'passed')
print(json.dumps(d['results']['stats']['failures']), 'failed')
print(d['results']['prompts'][0]['metrics']['namedScores'])
"

stats holds the counts, which is what a pass rate is built from. namedScores holds the metrics you named in lesson 11, one number each. Those two are what a dashboard reads; everything else in the file is detail for when a number looks wrong.

The other two formats

CSV is one row per result, for the people who want to sort and filter in a spreadsheet. It is the natural partner of the CSV tests from lesson 7: questions go in as a spreadsheet, answers come out as one.

JUnit XML is the format every build server already understands. Write it and your evals appear in the same test report as your unit tests, with names and failure messages, which is the shortest path to making them visible to a team that does not run promptfoo.

Write the file even when you are not reading it. A failed build with a results file is something you can open; a failed build with a scrolled-away table is something you have to reproduce. In CI it costs nothing to add -o results.json and keep it as an artifact.
Try it yourself
  • Write results.csv as well and open it in a spreadsheet.
  • Add a second named metric and watch it appear in namedScores.
  • Find the reason of a failed assertion inside the JSON.

This is what real progress feels like.