A config file and one command that runs your prompts and says which answers broke. Built by promptfoo.
Others in evals and testing
Common questions
Is Promptfoo free to use?
Promptfoo is open source and free to run yourself. You still pay whichever model provider you point it at, and this tutorial is free with no signup.
Do I need to know Python to use Promptfoo?
Basic Python is enough. Functions, dictionaries and imports cover most of what Promptfoo asks of you.
When should I not use Promptfoo?
When something else in evals and testing fits the job better. The others in that group are listed below.
Lessons for Promptfoo are being written. Meanwhile the Claude Code tutorial covers the same ground: state, tools, loops, memory and human approval. Most of it carries straight over.
Start the Claude Code tutorial →