Jailbreaking, in the context of AI, is the use of crafted prompts or techniques to get a large language model to bypass its safety guardrails and produce output it is designed to refuse. Common methods include role-play, hypothetical framing, encoding requests in formats such as Base64, and step-by-step pretexting that hides the real intent.
Why it matters: Safety guardrails are only as good as their resistance to people trying to get around them. A model that refuses a direct harmful request but complies when it is wrapped in a story is not safe in practice. Jailbreak testing is central to evaluating whether an AI system is harmless, and to protecting the organisation that deploys it.
In practice: Testers and red teams build collections of jailbreak techniques and apply them to a model's known refusal areas, such as weapons, malware or private data. They track which techniques succeed, how often, and whether fixes hold across model updates. Automated tools can generate many prompt variations, while human testers explore creative approaches that tools miss. Findings feed back into prompts, filters and training.
Examples:
- Asking a model to play a character with "no rules", such as the well-known DAN prompt, to unlock restricted answers.
- Encoding a harmful request in Base64 to slip past keyword-based filters.
- Building up to a dangerous request over several turns so no single message looks harmful.
Common pitfalls:
- Testing only direct harmful requests and concluding the model is safe because it refuses them.
- Treating a fixed jailbreak as solved, when small variations of the same technique often still work.
What is the difference between jailbreaking and prompt injection? Jailbreaking is a user trying to bypass a model's own safety rules. Prompt injection hides malicious instructions in content the model processes, such as a web page, to hijack its behaviour.
Is jailbreak testing legal? Testing your own or authorised systems is a normal part of AI safety work. Testing other providers' systems should follow their terms and any responsible disclosure programmes.
