Garakgarak 0.17.0 Ā· Python 3.10+
0%
1
Curious builder0 XP earned Ā· 300 to level 2
0 daysFinish a lesson to begin
Badge collection0 of 6 unlocked
23 small wins to finish your pathNext lesson →

--spec: choosing what runs

Every scan so far named one probe. --spec is the one argument that chooses probes, buffs and whole families, and it takes more than a name.

Written asMeans
probes.goodsideevery active probe in the family
probes.goodside.Tagthat one class
probes.a,probes.bboth, comma separated
-probes.goodside.Tageverything except that one
tag:owasp:llm01every probe carrying that tag
buffs.lowercase.Lowercasea buff, which lesson 17 covers

The minus is the useful one. A family is usually right except for the one probe that takes ten minutes, and saying so is shorter than listing the rest.

Example
garak -t function -n support_bot#reply --spec probes.goodside,-probes.goodside.Tag --generations 1 --report_prefix sub 2>&1 | tail -5

Two probes instead of three. goodside.Tag was named and removed, and the run is shorter by the 32 prompts it would have sent.

The flags this replaced

--probes and --buffs both still work and both print a deprecation notice. They were separate arguments for things that are now one selection language, which is why most material you will find online uses them and why your output will differ from that material.

The old flags are the single biggest difference between this course and what you will read elsewhere. If a tutorial shows --model_type, --model_name and --probes, it predates 0.15 and everything it says about selection is written the old way.
Try it yourself
  • Run a family with two exclusions and count the probes in the queue line.
  • Try --spec tag:owasp:llm01 and see how many probes carry it.
  • Run with --probes instead and read the notice.

Slow is fine. Stopping is the only problem.