Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
51 changes: 51 additions & 0 deletions doc/code/memory/8_seed_database.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -228,6 +228,57 @@
"print(\"----------\")\n",
"print_group(seed_groups[0])"
]
},
{
"cell_type": "markdown",
"id": "05f3d1d3",
"metadata": {},
"source": [
"## Removing Seeds from the Database\n",
"\n",
"Just as you can add and query seeds, you can remove them using `remove_seeds_from_memory`. It accepts the same filter parameters as `get_seeds`, so the recommended workflow is to preview the matching seeds with `get_seeds(...)` first, then remove them with the same filters. The method returns the number of seeds removed.\n",
"\n",
"As a safety measure, at least one filter must be provided. Calling it with no filters raises a `ValueError` to prevent accidentally deleting the entire seed database."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "51a51a3b",
"metadata": {},
"outputs": [],
"source": [
"# Preview the seeds that will be removed using the same filters\n",
"seeds_to_remove = memory.get_seeds(dataset_name=\"pyrit_example_dataset\")\n",
"print(f\"Seeds matching the filter: {len(seeds_to_remove)}\")\n",
"\n",
"# Remove them and get back the number of seeds deleted\n",
"removed_count = memory.remove_seeds_from_memory(dataset_name=\"pyrit_example_dataset\")\n",
"print(f\"Removed {removed_count} seeds\")\n",
"\n",
"# Confirm they are gone\n",
"seeds = memory.get_seeds(dataset_name=\"pyrit_example_dataset\")\n",
"print(f\"Seeds remaining in dataset: {len(seeds)}\")"
]
},
{
"cell_type": "markdown",
"id": "7b2c9e14",
"metadata": {},
"source": [
"### Removing entire groups\n",
"\n",
"`remove_seeds_from_memory` deletes only the individual seeds that match your filters. Because a seed group (for example a multimodal prompt made of text plus an image, or a multi-turn conversation) is stored as several seeds sharing a `prompt_group_id`, filtering by a single modality or attribute can leave a **partial group** behind. Some consequences to be aware of:\n",
"\n",
"- Deleting the sole objective while leaving its prompts produces an invalid `AttackSeedGroup`, and scenario initialization will raise a `ValueError`.\n",
"- Deleting one modality (e.g. the image) leaves a group that is still valid but now sends only text.\n",
"- Deleting one turn of a multi-turn conversation leaves the group with an incomplete context.\n",
"- Deleting the only role-bearing prompt in a sequence can cause a surviving multi-sequence group to fail role validation.\n",
"\n",
"For the most part these are user errors, but when you want to remove whole groups rather than individual seeds, use `remove_seed_groups_from_memory`. It applies the same filters, but removes every seed that shares a `prompt_group_id` with any match, so groups are never left partial.\n",
"\n",
"> **Note on deleting by `value`.** The `value` filter matches by substring, so a call like `remove_seeds_from_memory(value=\"the\")` could match and delete far more seeds than intended. Always preview with `get_seeds(...)` using the same filters first, and prefer a more specific filter (such as `dataset_name` or `value_sha256`) when deleting. A stricter/safer matching mode for value-based deletion is left as follow-up work."
]
}
],
"metadata": {
Expand Down
34 changes: 34 additions & 0 deletions doc/code/memory/8_seed_database.py
Original file line number Diff line number Diff line change
Expand Up @@ -113,3 +113,37 @@ def print_group(seed_group):
seed_groups = memory.get_seed_groups(data_types=["image_path"], dataset_name="pyrit_example_dataset")
print("----------")
print_group(seed_groups[0])

# %% [markdown]
# ## Removing Seeds from the Database
#
# Just as you can add and query seeds, you can remove them using `remove_seeds_from_memory`. It accepts the same filter parameters as `get_seeds`, so the recommended workflow is to preview the matching seeds with `get_seeds(...)` first, then remove them with the same filters. The method returns the number of seeds removed.
#
# As a safety measure, at least one filter must be provided. Calling it with no filters raises a `ValueError` to prevent accidentally deleting the entire seed database.

# %%
# Preview the seeds that will be removed using the same filters
seeds_to_remove = memory.get_seeds(dataset_name="pyrit_example_dataset")
print(f"Seeds matching the filter: {len(seeds_to_remove)}")

# Remove them and get back the number of seeds deleted
removed_count = memory.remove_seeds_from_memory(dataset_name="pyrit_example_dataset")
print(f"Removed {removed_count} seeds")

# Confirm they are gone
seeds = memory.get_seeds(dataset_name="pyrit_example_dataset")
print(f"Seeds remaining in dataset: {len(seeds)}")

# %% [markdown]
# ### Removing entire groups
#
# `remove_seeds_from_memory` deletes only the individual seeds that match your filters. Because a seed group (for example a multimodal prompt made of text plus an image, or a multi-turn conversation) is stored as several seeds sharing a `prompt_group_id`, filtering by a single modality or attribute can leave a **partial group** behind. Some consequences to be aware of:
#
# - Deleting the sole objective while leaving its prompts produces an invalid `AttackSeedGroup`, and scenario initialization will raise a `ValueError`.
# - Deleting one modality (e.g. the image) leaves a group that is still valid but now sends only text.
# - Deleting one turn of a multi-turn conversation leaves the group with an incomplete context.
# - Deleting the only role-bearing prompt in a sequence can cause a surviving multi-sequence group to fail role validation.
#
# For the most part these are user errors, but when you want to remove whole groups rather than individual seeds, use `remove_seed_groups_from_memory`. It applies the same filters, but removes every seed that shares a `prompt_group_id` with any match, so groups are never left partial.
#
# > **Note on deleting by `value`.** The `value` filter matches by substring, so a call like `remove_seeds_from_memory(value="the")` could match and delete far more seeds than intended. Always preview with `get_seeds(...)` using the same filters first, and prefer a more specific filter (such as `dataset_name` or `value_sha256`) when deleting. A stricter/safer matching mode for value-based deletion is left as follow-up work.
Loading
Loading