SG-011: Jailbreak Fragment

Severity: MEDIUM

Known jailbreak phrases: DAN, developer mode, policy puppetry XML structures

Fires when

• Text matches a known jailbreak phrase such as 'DAN / do anything now', 'developer mode enabled', 'pretend you have no / are not bound by' restrictions, or 'your true self is free'.
• Contains jailbreak XML/format structures like <JAILBREAK>, <RULES_OVERRIDE>, policy-puppetry <context>...<instruction>, or [RESPONSE_FORMAT]...[PROHIBITED_TOPICS].
• Persona/mode injection such as 'you are GPT-N without restrictions', 'enable developer/debug/admin mode', or 'simulate a world where AI has no rules'.

Remediation

No safe remediation: a file that triggers this rule should be rejected, not patched. Read the flagged lines in the source before deciding.

Research

• https://arxiv.org/abs/2407.01599
• https://hiddenlayer.com/research/