Keeping seeds in memory

A dataset loaded from a file lives only in the variable holding it. Writing it to memory means the same database that records runs also holds the probes those runs came from, queryable by name.

Adding a dataset

Example

db = await arena()
dataset = SeedDataset.from_yaml_file("shop_leaks.prompt")
await db.add_seed_datasets_to_memory_async(datasets=[dataset], added_by="course")
print(db.get_seed_dataset_names())
print([seed.value for seed in db.get_seeds(dataset_name="shop_leaks")])

add_seed_datasets_to_memory_async stores every seed, tagged with who added it. get_seed_dataset_names lists the datasets in the database, and get_seeds reads one back by name. The probes are now data in the same store as the results, not lines in a script.

Adding it twice

Example

db = await arena()
dataset = SeedDataset.from_yaml_file("shop_leaks.prompt")
await db.add_seed_datasets_to_memory_async(datasets=[dataset], added_by="course")
await db.add_seed_datasets_to_memory_async(datasets=[dataset], added_by="course")
print(len(db.get_seeds(dataset_name="shop_leaks")), "seeds")

Still three. PyRIT recognises seeds it already holds by their content and does not store a second copy, so a script that loads its datasets on every run does not fill the database with duplicates.

Try it yourself

Add a second dataset from a different file and list the names.
Read the seeds back and pass their values to an attack.
Change one objective's text, add the file again, and count the seeds.

PreviousSeed groups and seed datasets

This is what real progress feels like.