Buffs: changing the prompt on the way out

A buff sits between the probe and the target and rewrites every prompt. It is how one probe becomes several, without writing new prompts.

Buffs are named in --spec, beside the probes. The older --buffs flag still works and prints a deprecation notice.

Example

garak -t function -n support_bot#reply --spec probes.test.Blank,buffs.lowercase.Lowercase --generations 1 --report_prefix b 2>&1 | grep -E 'loading buff|queue of probes|ok on'

The loading line is the confirmation. Every prompt the probe produced went through the buff before it reached the bot.

What ships

Buff	What it does to the prompt
lowercase.Lowercase	lower-cases it
encoding.Base64	encodes it
encoding.CharCode	turns it into character codes
low_resource_languages.LRLBuff	translates it into a less common language
paraphrase.Fast	rewrites it with a model, which it downloads

The encoding and translation buffs are the interesting ones, because they are the technique rather than the payload: the same request, dressed so that a filter reading English does not recognise it.

A buff multiplies the run. Every prompt becomes a prompt per buff, so two buffs on a 160 prompt probe is 480 prompts before generations are counted. Add them one at a time and watch the totals.

Try it yourself

Run the same probe with and without the buff and compare the attempt counts in the report.
Add buffs.encoding.Base64 and read what reached the bot with test.Repeat.
Work out which buff would test a filter that only reads English.

PreviousAttack success rate

You understood something today that you didn't yesterday.