A jailbreak of your public chatbot goes viral on social media, with screenshots of it saying offensive things. What do you do?

30-second answerSay your answer out loud first, then reveal.

Run it as an incident. Reproduce and contain quickly (block the specific pattern, tighten output filtering for that category, or temporarily restrict the feature), assess whether it went beyond embarrassing text (data leaks, actions), communicate through the right channels, then fix the root cause with layered defences and add the attack and its variants to regression tests. Afterwards, do a post-mortem and improve monitoring so similar attacks are detected early.

Timeline

First hours: contain.
• Reproduce the jailbreak; find the vulnerable path (system prompt, model, missing output filter).
• Hotfix: input pattern detection for the attack family, a stricter output classifier threshold for that category, or a temporary block of the feature.
• Check the logs for how many users tried it and whether anything worse happened (PII, tool misuse).
Communication: coordinate with comms/PR and legal; a factual, brief public response if appropriate; internal update.
Days: robust fix.
• Generalise beyond the exact prompt (attackers will create variants). Use automated generation of variants to test the fix.
• Consider model-level changes (a different model, a safety-tuned system prompt) plus output classifiers as a backstop.
Post-mortem: why didn't red-teaming catch it? Add the category to the red-team plan; add monitoring (spikes in flagged outputs, social listening).

Key principle: output classifiers are a crucial backstop for public bots, because model alignment alone will eventually be bypassed.

PreviousHow do you run a red-teaming program for an LLM application?

This is what real progress feels like.

A jailbreak of your public chatbot goes viral on social media, with screenshots of it saying offensive things. What do you do?

Timeline

Related