Where a list of prompts stops
The loop in lesson 1 had three parts, and garak has a name for each of them. Learning the four words now makes every message it prints readable.
| Your loop | Garak's name | What it is |
|---|---|---|
| The function being asked | generator | Anything that takes a prompt and gives back text |
| Your list of prompts | probe | A class that produces prompts for one weakness |
| The substring check | detector | A class that scores an answer between 0 and 1 |
| One prompt and its answer | attempt | Plus the detector scores, kept in the report |
The split matters more than the vocabulary. A probe does not know how its answers will be judged, and a detector does not know which prompt produced them, so the same detector serves many probes and a probe can be scored by several detectors at once.
The part that is not obvious
A detector returns a number per answer, where higher means worse. Garak reports the opposite, as the share that passed, and calls the failures hits. A hit is garak finding something, which is a success for the scan and a problem for you.
What garak brings that your list did not
Prompts you did not write. Around 190 probe classes ship with it, grouped into 42 families, from encoding tricks to known model quirks to content risks.
Judgements you did not write. Around 117 detectors, some of them string matching and some of them models.
A record. Every attempt is written to a report file as one JSON object per line, which is lesson 14.
A number. The attack success rate, which is what goes in a build or a report to a customer.
- Look at the table again and name each part of your lesson 1 loop out loud.
- Decide, before reading on, whether a hit should make a build fail.
- Write down one weakness of your own assistant that no string check could catch.
You understood something today that you didn't yesterday.