A standard benchmark for red teaming, so two results can be compared. Built by Center for AI Safety.
Others in red teaming
GarakScan your own AI system for known weaknesses.
PyRITProbe your own AI system, and keep the evidence.
DeepTeam
GiskardCommon questions
Is HarmBench free to use?
HarmBench is open source and free to run yourself. You still pay whichever model provider you point it at, and this tutorial is free with no signup.
Do I need to know Python to use HarmBench?
Basic Python is enough. Functions, dictionaries and imports cover most of what HarmBench asks of you.
When should I not use HarmBench?
When something else in red teaming fits the job better. The others in that group are listed below.
Lessons for HarmBench are being written. Meanwhile the APIs for AI tutorial covers the same ground: state, tools, loops, memory and human approval. Most of it carries straight over.
Start the APIs for AI tutorial →