NUWA RESEARCH RECORD
Agent Security Bench (ASB): Formalizing and Benchmarking Attacks and Defenses in LLM-based Agents
Original publication ↗ · BibTeX · RISThe captured revision reports vulnerability across multiple interfaces and limited protection from the evaluated defenses. Utility and security must be assessed together.
Research question
Which agent interfaces expose distinct attack and defense failure modes?
Design & assumptions
ASB separates system/user prompts, tool responses and retrieved memory, then evaluates attacks and defenses on a shared agent framework.
Findings
The captured revision reports vulnerability across multiple interfaces and limited protection from the evaluated defenses. Utility and security must be assessed together.
Limits & counterevidence
Attack surfaces, backbones, and defense configurations differ; its aggregate attack rate is not a deployment probability. Revision-dependent method and metric counts differ from older abstracts.
Source evidence
| System | Measure | Reported value | Conditions & location |
|---|---|---|---|
| Gemma2-9B | Attack success rate | 87.10%% | DPI#S5.T5; row Gemma2-9B; condition DPI; column Attack success rate |
| Gemma2-9B | Refusal rate | 4.30%% | DPI#S5.T5; row Gemma2-9B; condition DPI; column Refusal rate |
| Gemma2-9B | Attack success rate | 14.20%% | IPI#S5.T5; row Gemma2-9B; condition IPI; column Attack success rate |
| Gemma2-9B | Refusal rate | 15.00%% | IPI#S5.T5; row Gemma2-9B; condition IPI; column Refusal rate |
| Gemma2-9B | Attack success rate | 6.85%% | Memory poisoning#S5.T5; row Gemma2-9B; condition Memory poisoning; column Attack success rate |
| Gemma2-9B | Refusal rate | 9.85%% | Memory poisoning#S5.T5; row Gemma2-9B; condition Memory poisoning; column Refusal rate |
| Gemma2-9B | Attack success rate | 92.17%% | Mixed attack#S5.T5; row Gemma2-9B; condition Mixed attack; column Attack success rate |
| Gemma2-9B | Refusal rate | 1.33%% | Mixed attack#S5.T5; row Gemma2-9B; condition Mixed attack; column Refusal rate |
| Gemma2-9B | Attack success rate | 39.75%% | PoT backdoor#S5.T5; row Gemma2-9B; condition PoT backdoor; column Attack success rate |
| Gemma2-9B | Refusal rate | 5.25%% | PoT backdoor#S5.T5; row Gemma2-9B; condition PoT backdoor; column Refusal rate |
| Gemma2-9B | Attack success rate | 48.01%% | Average across attack types#S5.T5; row Gemma2-9B; condition Average across attack types; column Attack success rate |
| Gemma2-9B | Refusal rate | 7.15%% | Average across attack types#S5.T5; row Gemma2-9B; condition Average across attack types; column Refusal rate |
| Gemma2-27B | Attack success rate | 96.75%% | DPI#S5.T5; row Gemma2-27B; condition DPI; column Attack success rate |
| Gemma2-27B | Refusal rate | 0.90%% | DPI#S5.T5; row Gemma2-27B; condition DPI; column Refusal rate |
| Gemma2-27B | Attack success rate | 14.20%% | IPI#S5.T5; row Gemma2-27B; condition IPI; column Attack success rate |
| Gemma2-27B | Refusal rate | 3.90%% | IPI#S5.T5; row Gemma2-27B; condition IPI; column Refusal rate |
| Gemma2-27B | Attack success rate | 6.25%% | Memory poisoning#S5.T5; row Gemma2-27B; condition Memory poisoning; column Attack success rate |
| Gemma2-27B | Refusal rate | 5.45%% | Memory poisoning#S5.T5; row Gemma2-27B; condition Memory poisoning; column Refusal rate |
| Gemma2-27B | Attack success rate | 100.00%% | Mixed attack#S5.T5; row Gemma2-27B; condition Mixed attack; column Attack success rate |
| Gemma2-27B | Refusal rate | 0.50%% | Mixed attack#S5.T5; row Gemma2-27B; condition Mixed attack; column Refusal rate |
| Gemma2-27B | Attack success rate | 54.50%% | PoT backdoor#S5.T5; row Gemma2-27B; condition PoT backdoor; column Attack success rate |
| Gemma2-27B | Refusal rate | 3.50%% | PoT backdoor#S5.T5; row Gemma2-27B; condition PoT backdoor; column Refusal rate |
| Gemma2-27B | Attack success rate | 54.34%% | Average across attack types#S5.T5; row Gemma2-27B; condition Average across attack types; column Attack success rate |
| Gemma2-27B | Refusal rate | 2.85%% | Average across attack types#S5.T5; row Gemma2-27B; condition Average across attack types; column Refusal rate |
| LLaMA3-8B | Attack success rate | 25.20%% | DPI#S5.T5; row LLaMA3-8B; condition DPI; column Attack success rate |
| LLaMA3-8B | Refusal rate | 7.45%% | DPI#S5.T5; row LLaMA3-8B; condition DPI; column Refusal rate |
| LLaMA3-8B | Attack success rate | 10.55%% | IPI#S5.T5; row LLaMA3-8B; condition IPI; column Attack success rate |
| LLaMA3-8B | Refusal rate | 3.00%% | IPI#S5.T5; row LLaMA3-8B; condition IPI; column Refusal rate |
| LLaMA3-8B | Attack success rate | 3.30%% | Memory poisoning#S5.T5; row LLaMA3-8B; condition Memory poisoning; column Attack success rate |
| LLaMA3-8B | Refusal rate | 5.45%% | Memory poisoning#S5.T5; row LLaMA3-8B; condition Memory poisoning; column Refusal rate |
| LLaMA3-8B | Attack success rate | 40.75%% | Mixed attack#S5.T5; row LLaMA3-8B; condition Mixed attack; column Attack success rate |
| LLaMA3-8B | Refusal rate | 5.75%% | Mixed attack#S5.T5; row LLaMA3-8B; condition Mixed attack; column Refusal rate |
| LLaMA3-8B | Attack success rate | 21.50%% | PoT backdoor#S5.T5; row LLaMA3-8B; condition PoT backdoor; column Attack success rate |
| LLaMA3-8B | Refusal rate | 2.50%% | PoT backdoor#S5.T5; row LLaMA3-8B; condition PoT backdoor; column Refusal rate |
| LLaMA3-8B | Attack success rate | 20.26%% | Average across attack types#S5.T5; row LLaMA3-8B; condition Average across attack types; column Attack success rate |
| LLaMA3-8B | Refusal rate | 4.83%% | Average across attack types#S5.T5; row LLaMA3-8B; condition Average across attack types; column Refusal rate |
| LLaMA3-70B | Attack success rate | 86.15%% | DPI#S5.T5; row LLaMA3-70B; condition DPI; column Attack success rate |
| LLaMA3-70B | Refusal rate | 7.80%% | DPI#S5.T5; row LLaMA3-70B; condition DPI; column Refusal rate |
| LLaMA3-70B | Attack success rate | 43.70%% | IPI#S5.T5; row LLaMA3-70B; condition IPI; column Attack success rate |
| LLaMA3-70B | Refusal rate | 3.00%% | IPI#S5.T5; row LLaMA3-70B; condition IPI; column Refusal rate |
| LLaMA3-70B | Attack success rate | 1.85%% | Memory poisoning#S5.T5; row LLaMA3-70B; condition Memory poisoning; column Attack success rate |
| LLaMA3-70B | Refusal rate | 1.80%% | Memory poisoning#S5.T5; row LLaMA3-70B; condition Memory poisoning; column Refusal rate |
| LLaMA3-70B | Attack success rate | 85.50%% | Mixed attack#S5.T5; row LLaMA3-70B; condition Mixed attack; column Attack success rate |
| LLaMA3-70B | Refusal rate | 6.50%% | Mixed attack#S5.T5; row LLaMA3-70B; condition Mixed attack; column Refusal rate |
| LLaMA3-70B | Attack success rate | 57.00%% | PoT backdoor#S5.T5; row LLaMA3-70B; condition PoT backdoor; column Attack success rate |
| LLaMA3-70B | Refusal rate | 2.00%% | PoT backdoor#S5.T5; row LLaMA3-70B; condition PoT backdoor; column Refusal rate |
| LLaMA3-70B | Attack success rate | 54.84%% | Average across attack types#S5.T5; row LLaMA3-70B; condition Average across attack types; column Attack success rate |
| LLaMA3-70B | Refusal rate | 4.22%% | Average across attack types#S5.T5; row LLaMA3-70B; condition Average across attack types; column Refusal rate |
| LLaMA3.1-8B | Attack success rate | 51.10%% | DPI#S5.T5; row LLaMA3.1-8B; condition DPI; column Attack success rate |
| LLaMA3.1-8B | Refusal rate | 5.20%% | DPI#S5.T5; row LLaMA3.1-8B; condition DPI; column Refusal rate |
| LLaMA3.1-8B | Attack success rate | 6.40%% | IPI#S5.T5; row LLaMA3.1-8B; condition IPI; column Attack success rate |
| LLaMA3.1-8B | Refusal rate | 1.85%% | IPI#S5.T5; row LLaMA3.1-8B; condition IPI; column Refusal rate |
| LLaMA3.1-8B | Attack success rate | 25.65%% | Memory poisoning#S5.T5; row LLaMA3.1-8B; condition Memory poisoning; column Attack success rate |
| LLaMA3.1-8B | Refusal rate | 6.75%% | Memory poisoning#S5.T5; row LLaMA3.1-8B; condition Memory poisoning; column Refusal rate |
| LLaMA3.1-8B | Attack success rate | 73.50%% | Mixed attack#S5.T5; row LLaMA3.1-8B; condition Mixed attack; column Attack success rate |
| LLaMA3.1-8B | Refusal rate | 3.50%% | Mixed attack#S5.T5; row LLaMA3.1-8B; condition Mixed attack; column Refusal rate |
| LLaMA3.1-8B | Attack success rate | 19.00%% | PoT backdoor#S5.T5; row LLaMA3.1-8B; condition PoT backdoor; column Attack success rate |
| LLaMA3.1-8B | Refusal rate | 5.75%% | PoT backdoor#S5.T5; row LLaMA3.1-8B; condition PoT backdoor; column Refusal rate |
| LLaMA3.1-8B | Attack success rate | 35.13%% | Average across attack types#S5.T5; row LLaMA3.1-8B; condition Average across attack types; column Attack success rate |
| LLaMA3.1-8B | Refusal rate | 4.61%% | Average across attack types#S5.T5; row LLaMA3.1-8B; condition Average across attack types; column Refusal rate |
| LLaMA3.1-70B | Attack success rate | 85.65%% | DPI#S5.T5; row LLaMA3.1-70B; condition DPI; column Attack success rate |
| LLaMA3.1-70B | Refusal rate | 5.30%% | DPI#S5.T5; row LLaMA3.1-70B; condition DPI; column Refusal rate |
| LLaMA3.1-70B | Attack success rate | 12.10%% | IPI#S5.T5; row LLaMA3.1-70B; condition IPI; column Attack success rate |
| LLaMA3.1-70B | Refusal rate | 4.95%% | IPI#S5.T5; row LLaMA3.1-70B; condition IPI; column Refusal rate |
| LLaMA3.1-70B | Attack success rate | 2.85%% | Memory poisoning#S5.T5; row LLaMA3.1-70B; condition Memory poisoning; column Attack success rate |
| LLaMA3.1-70B | Refusal rate | 2.20%% | Memory poisoning#S5.T5; row LLaMA3.1-70B; condition Memory poisoning; column Refusal rate |
| LLaMA3.1-70B | Attack success rate | 94.50%% | Mixed attack#S5.T5; row LLaMA3.1-70B; condition Mixed attack; column Attack success rate |
| LLaMA3.1-70B | Refusal rate | 1.25%% | Mixed attack#S5.T5; row LLaMA3.1-70B; condition Mixed attack; column Refusal rate |
| LLaMA3.1-70B | Attack success rate | 59.75%% | PoT backdoor#S5.T5; row LLaMA3.1-70B; condition PoT backdoor; column Attack success rate |
| LLaMA3.1-70B | Refusal rate | 6.25%% | PoT backdoor#S5.T5; row LLaMA3.1-70B; condition PoT backdoor; column Refusal rate |
| LLaMA3.1-70B | Attack success rate | 50.97%% | Average across attack types#S5.T5; row LLaMA3.1-70B; condition Average across attack types; column Attack success rate |
| LLaMA3.1-70B | Refusal rate | 3.99%% | Average across attack types#S5.T5; row LLaMA3.1-70B; condition Average across attack types; column Refusal rate |
| Mixtral-8x7B | Attack success rate | 25.85%% | DPI#S5.T5; row Mixtral-8x7B; condition DPI; column Attack success rate |
| Mixtral-8x7B | Refusal rate | 9.55%% | DPI#S5.T5; row Mixtral-8x7B; condition DPI; column Refusal rate |
| Mixtral-8x7B | Attack success rate | 4.80%% | IPI#S5.T5; row Mixtral-8x7B; condition IPI; column Attack success rate |
| Mixtral-8x7B | Refusal rate | 8.55%% | IPI#S5.T5; row Mixtral-8x7B; condition IPI; column Refusal rate |
| Mixtral-8x7B | Attack success rate | 4.90%% | Memory poisoning#S5.T5; row Mixtral-8x7B; condition Memory poisoning; column Attack success rate |
| Mixtral-8x7B | Refusal rate | 1.35%% | Memory poisoning#S5.T5; row Mixtral-8x7B; condition Memory poisoning; column Refusal rate |
| Mixtral-8x7B | Attack success rate | 54.75%% | Mixed attack#S5.T5; row Mixtral-8x7B; condition Mixed attack; column Attack success rate |
| Mixtral-8x7B | Refusal rate | 6.75%% | Mixed attack#S5.T5; row Mixtral-8x7B; condition Mixed attack; column Refusal rate |
| Mixtral-8x7B | Attack success rate | 4.75%% | PoT backdoor#S5.T5; row Mixtral-8x7B; condition PoT backdoor; column Attack success rate |
| Mixtral-8x7B | Refusal rate | 13.25%% | PoT backdoor#S5.T5; row Mixtral-8x7B; condition PoT backdoor; column Refusal rate |
| Mixtral-8x7B | Attack success rate | 19.01%% | Average across attack types#S5.T5; row Mixtral-8x7B; condition Average across attack types; column Attack success rate |
| Mixtral-8x7B | Refusal rate | 7.89%% | Average across attack types#S5.T5; row Mixtral-8x7B; condition Average across attack types; column Refusal rate |
| Qwen2-7B | Attack success rate | 55.20%% | DPI#S5.T5; row Qwen2-7B; condition DPI; column Attack success rate |
| Qwen2-7B | Refusal rate | 7.70%% | DPI#S5.T5; row Qwen2-7B; condition DPI; column Refusal rate |
| Qwen2-7B | Attack success rate | 9.00%% | IPI#S5.T5; row Qwen2-7B; condition IPI; column Attack success rate |
| Qwen2-7B | Refusal rate | 6.00%% | IPI#S5.T5; row Qwen2-7B; condition IPI; column Refusal rate |
| Qwen2-7B | Attack success rate | 2.85%% | Memory poisoning#S5.T5; row Qwen2-7B; condition Memory poisoning; column Attack success rate |
| Qwen2-7B | Refusal rate | 4.95%% | Memory poisoning#S5.T5; row Qwen2-7B; condition Memory poisoning; column Refusal rate |
| Qwen2-7B | Attack success rate | 76.00%% | Mixed attack#S5.T5; row Qwen2-7B; condition Mixed attack; column Attack success rate |
| Qwen2-7B | Refusal rate | 2.50%% | Mixed attack#S5.T5; row Qwen2-7B; condition Mixed attack; column Refusal rate |
| Qwen2-7B | Attack success rate | 12.25%% | PoT backdoor#S5.T5; row Qwen2-7B; condition PoT backdoor; column Attack success rate |
| Qwen2-7B | Refusal rate | 4.50%% | PoT backdoor#S5.T5; row Qwen2-7B; condition PoT backdoor; column Refusal rate |
| Qwen2-7B | Attack success rate | 31.06%% | Average across attack types#S5.T5; row Qwen2-7B; condition Average across attack types; column Attack success rate |
| Qwen2-7B | Refusal rate | 5.13%% | Average across attack types#S5.T5; row Qwen2-7B; condition Average across attack types; column Refusal rate |
| Qwen2-72B | Attack success rate | 86.95%% | DPI#S5.T5; row Qwen2-72B; condition DPI; column Attack success rate |
| Qwen2-72B | Refusal rate | 4.20%% | DPI#S5.T5; row Qwen2-72B; condition DPI; column Refusal rate |
| Qwen2-72B | Attack success rate | 21.35%% | IPI#S5.T5; row Qwen2-72B; condition IPI; column Attack success rate |
| Qwen2-72B | Refusal rate | 16.55%% | IPI#S5.T5; row Qwen2-72B; condition IPI; column Refusal rate |
| Qwen2-72B | Attack success rate | 3.95%% | Memory poisoning#S5.T5; row Qwen2-72B; condition Memory poisoning; column Attack success rate |
| Qwen2-72B | Refusal rate | 5.45%% | Memory poisoning#S5.T5; row Qwen2-72B; condition Memory poisoning; column Refusal rate |
| Qwen2-72B | Attack success rate | 98.50%% | Mixed attack#S5.T5; row Qwen2-72B; condition Mixed attack; column Attack success rate |
| Qwen2-72B | Refusal rate | 0.75%% | Mixed attack#S5.T5; row Qwen2-72B; condition Mixed attack; column Refusal rate |
| Qwen2-72B | Attack success rate | 57.75%% | PoT backdoor#S5.T5; row Qwen2-72B; condition PoT backdoor; column Attack success rate |
| Qwen2-72B | Refusal rate | 4.75%% | PoT backdoor#S5.T5; row Qwen2-72B; condition PoT backdoor; column Refusal rate |
| Qwen2-72B | Attack success rate | 53.70%% | Average across attack types#S5.T5; row Qwen2-72B; condition Average across attack types; column Attack success rate |
| Qwen2-72B | Refusal rate | 6.34%% | Average across attack types#S5.T5; row Qwen2-72B; condition Average across attack types; column Refusal rate |
| Claude3.5 Sonnet | Attack success rate | 90.75%% | DPI#S5.T5; row Claude3.5 Sonnet; condition DPI; column Attack success rate |
| Claude3.5 Sonnet | Refusal rate | 7.65%% | DPI#S5.T5; row Claude3.5 Sonnet; condition DPI; column Refusal rate |
| Claude3.5 Sonnet | Attack success rate | 59.70%% | IPI#S5.T5; row Claude3.5 Sonnet; condition IPI; column Attack success rate |
| Claude3.5 Sonnet | Refusal rate | 25.50%% | IPI#S5.T5; row Claude3.5 Sonnet; condition IPI; column Refusal rate |
| Claude3.5 Sonnet | Attack success rate | 19.75%% | Memory poisoning#S5.T5; row Claude3.5 Sonnet; condition Memory poisoning; column Attack success rate |
| Claude3.5 Sonnet | Refusal rate | 1.20%% | Memory poisoning#S5.T5; row Claude3.5 Sonnet; condition Memory poisoning; column Refusal rate |
| Claude3.5 Sonnet | Attack success rate | 94.50%% | Mixed attack#S5.T5; row Claude3.5 Sonnet; condition Mixed attack; column Attack success rate |
| Claude3.5 Sonnet | Refusal rate | 6.25%% | Mixed attack#S5.T5; row Claude3.5 Sonnet; condition Mixed attack; column Refusal rate |
| Claude3.5 Sonnet | Attack success rate | 17.50%% | PoT backdoor#S5.T5; row Claude3.5 Sonnet; condition PoT backdoor; column Attack success rate |
| Claude3.5 Sonnet | Refusal rate | 11.75%% | PoT backdoor#S5.T5; row Claude3.5 Sonnet; condition PoT backdoor; column Refusal rate |
| Claude3.5 Sonnet | Attack success rate | 56.44%% | Average across attack types#S5.T5; row Claude3.5 Sonnet; condition Average across attack types; column Attack success rate |
| Claude3.5 Sonnet | Refusal rate | 10.47%% | Average across attack types#S5.T5; row Claude3.5 Sonnet; condition Average across attack types; column Refusal rate |
| GPT-3.5 Turbo | Attack success rate | 98.40%% | DPI#S5.T5; row GPT-3.5 Turbo; condition DPI; column Attack success rate |
| GPT-3.5 Turbo | Refusal rate | 3.00%% | DPI#S5.T5; row GPT-3.5 Turbo; condition DPI; column Refusal rate |
| GPT-3.5 Turbo | Attack success rate | 55.10%% | IPI#S5.T5; row GPT-3.5 Turbo; condition IPI; column Attack success rate |
| GPT-3.5 Turbo | Refusal rate | 16.85%% | IPI#S5.T5; row GPT-3.5 Turbo; condition IPI; column Refusal rate |
| GPT-3.5 Turbo | Attack success rate | 9.30%% | Memory poisoning#S5.T5; row GPT-3.5 Turbo; condition Memory poisoning; column Attack success rate |
| GPT-3.5 Turbo | Refusal rate | 0.30%% | Memory poisoning#S5.T5; row GPT-3.5 Turbo; condition Memory poisoning; column Refusal rate |
| GPT-3.5 Turbo | Attack success rate | 99.75%% | Mixed attack#S5.T5; row GPT-3.5 Turbo; condition Mixed attack; column Attack success rate |
| GPT-3.5 Turbo | Refusal rate | 0.00%% | Mixed attack#S5.T5; row GPT-3.5 Turbo; condition Mixed attack; column Refusal rate |
| GPT-3.5 Turbo | Attack success rate | 8.25%% | PoT backdoor#S5.T5; row GPT-3.5 Turbo; condition PoT backdoor; column Attack success rate |
| GPT-3.5 Turbo | Refusal rate | 10.75%% | PoT backdoor#S5.T5; row GPT-3.5 Turbo; condition PoT backdoor; column Refusal rate |
| GPT-3.5 Turbo | Attack success rate | 54.16%% | Average across attack types#S5.T5; row GPT-3.5 Turbo; condition Average across attack types; column Attack success rate |
| GPT-3.5 Turbo | Refusal rate | 6.18%% | Average across attack types#S5.T5; row GPT-3.5 Turbo; condition Average across attack types; column Refusal rate |
| GPT-4o | Attack success rate | 60.35%% | DPI#S5.T5; row GPT-4o; condition DPI; column Attack success rate |
| GPT-4o | Refusal rate | 20.05%% | DPI#S5.T5; row GPT-4o; condition DPI; column Refusal rate |
| GPT-4o | Attack success rate | 62.45%% | IPI#S5.T5; row GPT-4o; condition IPI; column Attack success rate |
| GPT-4o | Refusal rate | 6.50%% | IPI#S5.T5; row GPT-4o; condition IPI; column Refusal rate |
| GPT-4o | Attack success rate | 10.00%% | Memory poisoning#S5.T5; row GPT-4o; condition Memory poisoning; column Attack success rate |
| GPT-4o | Refusal rate | 11.75%% | Memory poisoning#S5.T5; row GPT-4o; condition Memory poisoning; column Refusal rate |
| GPT-4o | Attack success rate | 89.25%% | Mixed attack#S5.T5; row GPT-4o; condition Mixed attack; column Attack success rate |
| GPT-4o | Refusal rate | 5.50%% | Mixed attack#S5.T5; row GPT-4o; condition Mixed attack; column Refusal rate |
| GPT-4o | Attack success rate | 100.00%% | PoT backdoor#S5.T5; row GPT-4o; condition PoT backdoor; column Attack success rate |
| GPT-4o | Refusal rate | 0.25%% | PoT backdoor#S5.T5; row GPT-4o; condition PoT backdoor; column Refusal rate |
| GPT-4o | Attack success rate | 64.41%% | Average across attack types#S5.T5; row GPT-4o; condition Average across attack types; column Attack success rate |
| GPT-4o | Refusal rate | 8.81%% | Average across attack types#S5.T5; row GPT-4o; condition Average across attack types; column Refusal rate |
| GPT-4o-mini | Attack success rate | 95.45%% | DPI#S5.T5; row GPT-4o-mini; condition DPI; column Attack success rate |
| GPT-4o-mini | Refusal rate | 1.85%% | DPI#S5.T5; row GPT-4o-mini; condition DPI; column Refusal rate |
| GPT-4o-mini | Attack success rate | 44.55%% | IPI#S5.T5; row GPT-4o-mini; condition IPI; column Attack success rate |
| GPT-4o-mini | Refusal rate | 0.25%% | IPI#S5.T5; row GPT-4o-mini; condition IPI; column Refusal rate |
| GPT-4o-mini | Attack success rate | 5.50%% | Memory poisoning#S5.T5; row GPT-4o-mini; condition Memory poisoning; column Attack success rate |
| GPT-4o-mini | Refusal rate | 3.65%% | Memory poisoning#S5.T5; row GPT-4o-mini; condition Memory poisoning; column Refusal rate |
| GPT-4o-mini | Attack success rate | 96.75%% | Mixed attack#S5.T5; row GPT-4o-mini; condition Mixed attack; column Attack success rate |
| GPT-4o-mini | Refusal rate | 1.25%% | Mixed attack#S5.T5; row GPT-4o-mini; condition Mixed attack; column Refusal rate |
| GPT-4o-mini | Attack success rate | 95.50%% | PoT backdoor#S5.T5; row GPT-4o-mini; condition PoT backdoor; column Attack success rate |
| GPT-4o-mini | Refusal rate | 0.00%% | PoT backdoor#S5.T5; row GPT-4o-mini; condition PoT backdoor; column Refusal rate |
| GPT-4o-mini | Attack success rate | 67.55%% | Average across attack types#S5.T5; row GPT-4o-mini; condition Average across attack types; column Attack success rate |
| GPT-4o-mini | Refusal rate | 1.40%% | Average across attack types#S5.T5; row GPT-4o-mini; condition Average across attack types; column Refusal rate |
| Average | Attack success rate | 72.68%% | DPI#S5.T5; row Average; condition DPI; column Attack success rate |
| Average | Refusal rate | 6.53%% | DPI#S5.T5; row Average; condition DPI; column Refusal rate |
| Average | Attack success rate | 27.55%% | IPI#S5.T5; row Average; condition IPI; column Attack success rate |
| Average | Refusal rate | 8.61%% | IPI#S5.T5; row Average; condition IPI; column Refusal rate |
| Average | Attack success rate | 7.92%% | Memory poisoning#S5.T5; row Average; condition Memory poisoning; column Attack success rate |
| Average | Refusal rate | 4.63%% | Memory poisoning#S5.T5; row Average; condition Memory poisoning; column Refusal rate |
| Average | Attack success rate | 84.30%% | Mixed attack#S5.T5; row Average; condition Mixed attack; column Attack success rate |
| Average | Refusal rate | 3.22%% | Mixed attack#S5.T5; row Average; condition Mixed attack; column Refusal rate |
| Average | Attack success rate | 42.12%% | PoT backdoor#S5.T5; row Average; condition PoT backdoor; column Attack success rate |
| Average | Refusal rate | 5.42%% | PoT backdoor#S5.T5; row Average; condition PoT backdoor; column Refusal rate |
| Average | Attack success rate | 46.91%% | Average across attack types#S5.T5; row Average; condition Average across attack types; column Attack success rate |
| Average | Refusal rate | 5.68%% | Average across attack types#S5.T5; row Average; condition Average across attack types; column Refusal rate |
| Claude-3.5 Sonnet | Performance under no attack | 100.00% | No attack#S5.T6; row Claude-3.5 Sonnet; condition No attack; column Performance under no attack |
| Claude-3.5 Sonnet | Attack success rate | 56.44% | Average across attack types#S5.T6; row Claude-3.5 Sonnet; condition Average across attack types; column Attack success rate |
| Claude-3.5 Sonnet | NRP | 43.56% | Normalized risk-adjusted performance#S5.T6; row Claude-3.5 Sonnet; condition Normalized risk-adjusted performance; column NRP |
| LLaMA3-70B | Performance under no attack | 66.50% | No attack#S5.T6; row LLaMA3-70B; condition No attack; column Performance under no attack |
| LLaMA3-70B | Attack success rate | 54.84% | Average across attack types#S5.T6; row LLaMA3-70B; condition Average across attack types; column Attack success rate |
| LLaMA3-70B | NRP | 30.03% | Normalized risk-adjusted performance#S5.T6; row LLaMA3-70B; condition Normalized risk-adjusted performance; column NRP |
| GPT-4o | Performance under no attack | 79.00% | No attack#S5.T6; row GPT-4o; condition No attack; column Performance under no attack |
| GPT-4o | Attack success rate | 64.41% | Average across attack types#S5.T6; row GPT-4o; condition Average across attack types; column Attack success rate |
| GPT-4o | NRP | 28.12% | Normalized risk-adjusted performance#S5.T6; row GPT-4o; condition Normalized risk-adjusted performance; column NRP |
| GPT-4o-mini | Performance under no attack | 50.00% | No attack#S5.T6; row GPT-4o-mini; condition No attack; column Performance under no attack |
| GPT-4o-mini | Attack success rate | 67.55% | Average across attack types#S5.T6; row GPT-4o-mini; condition Average across attack types; column Attack success rate |
| GPT-4o-mini | NRP | 16.23% | Normalized risk-adjusted performance#S5.T6; row GPT-4o-mini; condition Normalized risk-adjusted performance; column NRP |
| Gemma2-27B | Performance under no attack | 31.50% | No attack#S5.T6; row Gemma2-27B; condition No attack; column Performance under no attack |
| Gemma2-27B | Attack success rate | 54.34% | Average across attack types#S5.T6; row Gemma2-27B; condition Average across attack types; column Attack success rate |
| Gemma2-27B | NRP | 14.38% | Normalized risk-adjusted performance#S5.T6; row Gemma2-27B; condition Normalized risk-adjusted performance; column NRP |
| LLaMA3.1-70B | Performance under no attack | 21.25% | No attack#S5.T6; row LLaMA3.1-70B; condition No attack; column Performance under no attack |
| LLaMA3.1-70B | Attack success rate | 50.97% | Average across attack types#S5.T6; row LLaMA3.1-70B; condition Average across attack types; column Attack success rate |
| LLaMA3.1-70B | NRP | 10.42% | Normalized risk-adjusted performance#S5.T6; row LLaMA3.1-70B; condition Normalized risk-adjusted performance; column NRP |
| Qwen2-7B | Performance under no attack | 9.75% | No attack#S5.T6; row Qwen2-7B; condition No attack; column Performance under no attack |
| Qwen2-7B | Attack success rate | 31.06% | Average across attack types#S5.T6; row Qwen2-7B; condition Average across attack types; column Attack success rate |
| Qwen2-7B | NRP | 6.72% | Normalized risk-adjusted performance#S5.T6; row Qwen2-7B; condition Normalized risk-adjusted performance; column NRP |
| Gemma2-9B | Performance under no attack | 10.75% | No attack#S5.T6; row Gemma2-9B; condition No attack; column Performance under no attack |
| Gemma2-9B | Attack success rate | 48.01% | Average across attack types#S5.T6; row Gemma2-9B; condition Average across attack types; column Attack success rate |
| Gemma2-9B | NRP | 5.59% | Normalized risk-adjusted performance#S5.T6; row Gemma2-9B; condition Normalized risk-adjusted performance; column NRP |
| GPT-3.5 Turbo | Performance under no attack | 8.00% | No attack#S5.T6; row GPT-3.5 Turbo; condition No attack; column Performance under no attack |
| GPT-3.5 Turbo | Attack success rate | 54.16% | Average across attack types#S5.T6; row GPT-3.5 Turbo; condition Average across attack types; column Attack success rate |
| GPT-3.5 Turbo | NRP | 3.67% | Normalized risk-adjusted performance#S5.T6; row GPT-3.5 Turbo; condition Normalized risk-adjusted performance; column NRP |
| Qwen2-72B | Performance under no attack | 4.00% | No attack#S5.T6; row Qwen2-72B; condition No attack; column Performance under no attack |
| Qwen2-72B | Attack success rate | 53.70% | Average across attack types#S5.T6; row Qwen2-72B; condition Average across attack types; column Attack success rate |
| Qwen2-72B | NRP | 1.85% | Normalized risk-adjusted performance#S5.T6; row Qwen2-72B; condition Normalized risk-adjusted performance; column NRP |
| LLaMA3-8B | Performance under no attack | 1.50% | No attack#S5.T6; row LLaMA3-8B; condition No attack; column Performance under no attack |
| LLaMA3-8B | Attack success rate | 20.26% | Average across attack types#S5.T6; row LLaMA3-8B; condition Average across attack types; column Attack success rate |
| LLaMA3-8B | NRP | 1.20% | Normalized risk-adjusted performance#S5.T6; row LLaMA3-8B; condition Normalized risk-adjusted performance; column NRP |
| LLaMA3.1-8B | Performance under no attack | 0.75% | No attack#S5.T6; row LLaMA3.1-8B; condition No attack; column Performance under no attack |
| LLaMA3.1-8B | Attack success rate | 35.13% | Average across attack types#S5.T6; row LLaMA3.1-8B; condition Average across attack types; column Attack success rate |
| LLaMA3.1-8B | NRP | 0.49% | Normalized risk-adjusted performance#S5.T6; row LLaMA3.1-8B; condition Normalized risk-adjusted performance; column NRP |
| Mixtral-8x7B | Performance under no attack | 0.00% | No attack#S5.T6; row Mixtral-8x7B; condition No attack; column Performance under no attack |
| Mixtral-8x7B | Attack success rate | 19.01% | Average across attack types#S5.T6; row Mixtral-8x7B; condition Average across attack types; column Attack success rate |
| Mixtral-8x7B | NRP | 0.00% | Normalized risk-adjusted performance#S5.T6; row Mixtral-8x7B; condition Normalized risk-adjusted performance; column NRP |
| Average | Performance under no attack | 29.46% | No attack#S5.T6; row Average; condition No attack; column Performance under no attack |
| Average | Attack success rate | 46.91% | Average across attack types#S5.T6; row Average; condition Average across attack types; column Attack success rate |
| Average | NRP | 15.64% | Normalized risk-adjusted performance#S5.T6; row Average; condition Normalized risk-adjusted performance; column NRP |
| Gemma2-9B | Attack success rate | 91.00%% | DPI baseline#S5.T8; row Gemma2-9B; condition DPI baseline; column Attack success rate |
| Gemma2-9B | Attack success rate | 91.75%% | Delimiter#S5.T8; row Gemma2-9B; condition Delimiter; column Attack success rate |
| Gemma2-9B | Attack success rate | 62.50%% | Paraphrase#S5.T8; row Gemma2-9B; condition Paraphrase; column Attack success rate |
| Gemma2-9B | Attack success rate | 91.00%% | Instructional#S5.T8; row Gemma2-9B; condition Instructional; column Attack success rate |
| Gemma2-9B | Attack success rate | 49.50%% | Dynamic prompt rewriting#S5.T8; row Gemma2-9B; condition Dynamic prompt rewriting; column Attack success rate |
| Gemma2-27B | Attack success rate | 98.75%% | DPI baseline#S5.T8; row Gemma2-27B; condition DPI baseline; column Attack success rate |
| Gemma2-27B | Attack success rate | 99.75%% | Delimiter#S5.T8; row Gemma2-27B; condition Delimiter; column Attack success rate |
| Gemma2-27B | Attack success rate | 68.00%% | Paraphrase#S5.T8; row Gemma2-27B; condition Paraphrase; column Attack success rate |
| Gemma2-27B | Attack success rate | 99.50%% | Instructional#S5.T8; row Gemma2-27B; condition Instructional; column Attack success rate |
| Gemma2-27B | Attack success rate | 67.50%% | Dynamic prompt rewriting#S5.T8; row Gemma2-27B; condition Dynamic prompt rewriting; column Attack success rate |
| LLaMA3-8B | Attack success rate | 33.75%% | DPI baseline#S5.T8; row LLaMA3-8B; condition DPI baseline; column Attack success rate |
| LLaMA3-8B | Attack success rate | 62.75%% | Delimiter#S5.T8; row LLaMA3-8B; condition Delimiter; column Attack success rate |
| LLaMA3-8B | Attack success rate | 28.50%% | Paraphrase#S5.T8; row LLaMA3-8B; condition Paraphrase; column Attack success rate |
| LLaMA3-8B | Attack success rate | 52.00%% | Instructional#S5.T8; row LLaMA3-8B; condition Instructional; column Attack success rate |
| LLaMA3-8B | Attack success rate | 23.15%% | Dynamic prompt rewriting#S5.T8; row LLaMA3-8B; condition Dynamic prompt rewriting; column Attack success rate |
| LLaMA3-70B | Attack success rate | 87.75%% | DPI baseline#S5.T8; row LLaMA3-70B; condition DPI baseline; column Attack success rate |
| LLaMA3-70B | Attack success rate | 88.25%% | Delimiter#S5.T8; row LLaMA3-70B; condition Delimiter; column Attack success rate |
| LLaMA3-70B | Attack success rate | 71.25%% | Paraphrase#S5.T8; row LLaMA3-70B; condition Paraphrase; column Attack success rate |
| LLaMA3-70B | Attack success rate | 87.25%% | Instructional#S5.T8; row LLaMA3-70B; condition Instructional; column Attack success rate |
| LLaMA3-70B | Attack success rate | 74.50%% | Dynamic prompt rewriting#S5.T8; row LLaMA3-70B; condition Dynamic prompt rewriting; column Attack success rate |
| LLaMA3.1-8B | Attack success rate | 64.25%% | DPI baseline#S5.T8; row LLaMA3.1-8B; condition DPI baseline; column Attack success rate |
| LLaMA3.1-8B | Attack success rate | 65.00%% | Delimiter#S5.T8; row LLaMA3.1-8B; condition Delimiter; column Attack success rate |
| LLaMA3.1-8B | Attack success rate | 42.50%% | Paraphrase#S5.T8; row LLaMA3.1-8B; condition Paraphrase; column Attack success rate |
| LLaMA3.1-8B | Attack success rate | 68.75%% | Instructional#S5.T8; row LLaMA3.1-8B; condition Instructional; column Attack success rate |
| LLaMA3.1-8B | Attack success rate | 26.70%% | Dynamic prompt rewriting#S5.T8; row LLaMA3.1-8B; condition Dynamic prompt rewriting; column Attack success rate |
| LLaMA3.1-70B | Attack success rate | 93.50%% | DPI baseline#S5.T8; row LLaMA3.1-70B; condition DPI baseline; column Attack success rate |
| LLaMA3.1-70B | Attack success rate | 92.75%% | Delimiter#S5.T8; row LLaMA3.1-70B; condition Delimiter; column Attack success rate |
| LLaMA3.1-70B | Attack success rate | 56.75%% | Paraphrase#S5.T8; row LLaMA3.1-70B; condition Paraphrase; column Attack success rate |
| LLaMA3.1-70B | Attack success rate | 90.50%% | Instructional#S5.T8; row LLaMA3.1-70B; condition Instructional; column Attack success rate |
| LLaMA3.1-70B | Attack success rate | 44.60%% | Dynamic prompt rewriting#S5.T8; row LLaMA3.1-70B; condition Dynamic prompt rewriting; column Attack success rate |
| Mixtral-8x7B | Attack success rate | 43.25%% | DPI baseline#S5.T8; row Mixtral-8x7B; condition DPI baseline; column Attack success rate |
| Mixtral-8x7B | Attack success rate | 43.00%% | Delimiter#S5.T8; row Mixtral-8x7B; condition Delimiter; column Attack success rate |
| Mixtral-8x7B | Attack success rate | 21.00%% | Paraphrase#S5.T8; row Mixtral-8x7B; condition Paraphrase; column Attack success rate |
| Mixtral-8x7B | Attack success rate | 34.00%% | Instructional#S5.T8; row Mixtral-8x7B; condition Instructional; column Attack success rate |
| Mixtral-8x7B | Attack success rate | 8.35%% | Dynamic prompt rewriting#S5.T8; row Mixtral-8x7B; condition Dynamic prompt rewriting; column Attack success rate |
| Qwen2-7B | Attack success rate | 73.50%% | DPI baseline#S5.T8; row Qwen2-7B; condition DPI baseline; column Attack success rate |
| Qwen2-7B | Attack success rate | 80.00%% | Delimiter#S5.T8; row Qwen2-7B; condition Delimiter; column Attack success rate |
| Qwen2-7B | Attack success rate | 46.25%% | Paraphrase#S5.T8; row Qwen2-7B; condition Paraphrase; column Attack success rate |
| Qwen2-7B | Attack success rate | 76.75%% | Instructional#S5.T8; row Qwen2-7B; condition Instructional; column Attack success rate |
| Qwen2-7B | Attack success rate | 33.30%% | Dynamic prompt rewriting#S5.T8; row Qwen2-7B; condition Dynamic prompt rewriting; column Attack success rate |
| Qwen2-72B | Attack success rate | 94.50%% | DPI baseline#S5.T8; row Qwen2-72B; condition DPI baseline; column Attack success rate |
| Qwen2-72B | Attack success rate | 95.00%% | Delimiter#S5.T8; row Qwen2-72B; condition Delimiter; column Attack success rate |
| Qwen2-72B | Attack success rate | 60.50%% | Paraphrase#S5.T8; row Qwen2-72B; condition Paraphrase; column Attack success rate |
| Qwen2-72B | Attack success rate | 95.50%% | Instructional#S5.T8; row Qwen2-72B; condition Instructional; column Attack success rate |
| Qwen2-72B | Attack success rate | 39.40%% | Dynamic prompt rewriting#S5.T8; row Qwen2-72B; condition Dynamic prompt rewriting; column Attack success rate |
| Claude-3.5 Sonnet | Attack success rate | 87.75%% | DPI baseline#S5.T8; row Claude-3.5 Sonnet; condition DPI baseline; column Attack success rate |
| Claude-3.5 Sonnet | Attack success rate | 79.00%% | Delimiter#S5.T8; row Claude-3.5 Sonnet; condition Delimiter; column Attack success rate |
| Claude-3.5 Sonnet | Attack success rate | 65.25%% | Paraphrase#S5.T8; row Claude-3.5 Sonnet; condition Paraphrase; column Attack success rate |
| Claude-3.5 Sonnet | Attack success rate | 70.25%% | Instructional#S5.T8; row Claude-3.5 Sonnet; condition Instructional; column Attack success rate |
| Claude-3.5 Sonnet | Attack success rate | 35.80%% | Dynamic prompt rewriting#S5.T8; row Claude-3.5 Sonnet; condition Dynamic prompt rewriting; column Attack success rate |
| GPT-3.5 Turbo | Attack success rate | 99.75%% | DPI baseline#S5.T8; row GPT-3.5 Turbo; condition DPI baseline; column Attack success rate |
| GPT-3.5 Turbo | Attack success rate | 99.75%% | Delimiter#S5.T8; row GPT-3.5 Turbo; condition Delimiter; column Attack success rate |
| GPT-3.5 Turbo | Attack success rate | 78.25%% | Paraphrase#S5.T8; row GPT-3.5 Turbo; condition Paraphrase; column Attack success rate |
| GPT-3.5 Turbo | Attack success rate | 99.50%% | Instructional#S5.T8; row GPT-3.5 Turbo; condition Instructional; column Attack success rate |
| GPT-3.5 Turbo | Attack success rate | 53.60%% | Dynamic prompt rewriting#S5.T8; row GPT-3.5 Turbo; condition Dynamic prompt rewriting; column Attack success rate |
| GPT-4o | Attack success rate | 55.50%% | DPI baseline#S5.T8; row GPT-4o; condition DPI baseline; column Attack success rate |
| GPT-4o | Attack success rate | 52.25%% | Delimiter#S5.T8; row GPT-4o; condition Delimiter; column Attack success rate |
| GPT-4o | Attack success rate | 62.50%% | Paraphrase#S5.T8; row GPT-4o; condition Paraphrase; column Attack success rate |
| GPT-4o | Attack success rate | 70.75%% | Instructional#S5.T8; row GPT-4o; condition Instructional; column Attack success rate |
| GPT-4o | Attack success rate | 60.20%% | Dynamic prompt rewriting#S5.T8; row GPT-4o; condition Dynamic prompt rewriting; column Attack success rate |
| GPT-4o-mini | Attack success rate | 95.75%% | DPI baseline#S5.T8; row GPT-4o-mini; condition DPI baseline; column Attack success rate |
| GPT-4o-mini | Attack success rate | 78.75%% | Delimiter#S5.T8; row GPT-4o-mini; condition Delimiter; column Attack success rate |
| GPT-4o-mini | Attack success rate | 76.00%% | Paraphrase#S5.T8; row GPT-4o-mini; condition Paraphrase; column Attack success rate |
| GPT-4o-mini | Attack success rate | 62.25%% | Instructional#S5.T8; row GPT-4o-mini; condition Instructional; column Attack success rate |
| GPT-4o-mini | Attack success rate | 61.25%% | Dynamic prompt rewriting#S5.T8; row GPT-4o-mini; condition Dynamic prompt rewriting; column Attack success rate |
| Average | Attack success rate | 78.38%% | DPI baseline#S5.T8; row Average; condition DPI baseline; column Attack success rate |
| Average | Attack success rate | 79.08%% | Delimiter#S5.T8; row Average; condition Delimiter; column Attack success rate |
| Average | Attack success rate | 56.87%% | Paraphrase#S5.T8; row Average; condition Paraphrase; column Attack success rate |
| Average | Attack success rate | 76.77%% | Instructional#S5.T8; row Average; condition Instructional; column Attack success rate |
| Average | Attack success rate | 44.45%% | Dynamic prompt rewriting#S5.T8; row Average; condition Dynamic prompt rewriting; column Attack success rate |
| Change versus table-specific baseline | Attack success rate | 0.00%percentage points | DPI baseline#S5.T8; row Δ; condition DPI baseline; column Attack success rate |
| Change versus table-specific baseline | Attack success rate | 0.69%percentage points | Delimiter#S5.T8; row Δ; condition Delimiter; column Attack success rate |
| Change versus table-specific baseline | Attack success rate | -21.52%percentage points | Paraphrase#S5.T8; row Δ; condition Paraphrase; column Attack success rate |
| Change versus table-specific baseline | Attack success rate | -1.62%percentage points | Instructional#S5.T8; row Δ; condition Instructional; column Attack success rate |
| Change versus table-specific baseline | Attack success rate | -33.93%percentage points | Dynamic prompt rewriting#S5.T8; row Δ; condition Dynamic prompt rewriting; column Attack success rate |
| Gemma2-9B | Attack success rate | 88.75%% | DPI+IPI#A4.T14; row Gemma2-9B; condition DPI+IPI; column Attack success rate |
| Gemma2-9B | Refusal rate | 3.75%% | DPI+IPI#A4.T14; row Gemma2-9B; condition DPI+IPI; column Refusal rate |
| Gemma2-9B | Attack success rate | 91.75%% | DPI+MP#A4.T14; row Gemma2-9B; condition DPI+MP; column Attack success rate |
| Gemma2-9B | Refusal rate | 1.75%% | DPI+MP#A4.T14; row Gemma2-9B; condition DPI+MP; column Refusal rate |
| Gemma2-9B | Attack success rate | 14.00%% | IPI+MP#A4.T14; row Gemma2-9B; condition IPI+MP; column Attack success rate |
| Gemma2-9B | Refusal rate | 26.75%% | IPI+MP#A4.T14; row Gemma2-9B; condition IPI+MP; column Refusal rate |
| Gemma2-9B | Attack success rate | 92.17%% | DPI+IPI+MP#A4.T14; row Gemma2-9B; condition DPI+IPI+MP; column Attack success rate |
| Gemma2-9B | Refusal rate | 1.33%% | DPI+IPI+MP#A4.T14; row Gemma2-9B; condition DPI+IPI+MP; column Refusal rate |
| Gemma2-27B | Attack success rate | 99.25%% | DPI+IPI#A4.T14; row Gemma2-27B; condition DPI+IPI; column Attack success rate |
| Gemma2-27B | Refusal rate | 0.00%% | DPI+IPI#A4.T14; row Gemma2-27B; condition DPI+IPI; column Refusal rate |
| Gemma2-27B | Attack success rate | 99.00%% | DPI+MP#A4.T14; row Gemma2-27B; condition DPI+MP; column Attack success rate |
| Gemma2-27B | Refusal rate | 1.00%% | DPI+MP#A4.T14; row Gemma2-27B; condition DPI+MP; column Refusal rate |
| Gemma2-27B | Attack success rate | 16.00%% | IPI+MP#A4.T14; row Gemma2-27B; condition IPI+MP; column Attack success rate |
| Gemma2-27B | Refusal rate | 3.25%% | IPI+MP#A4.T14; row Gemma2-27B; condition IPI+MP; column Refusal rate |
| Gemma2-27B | Attack success rate | 100.00%% | DPI+IPI+MP#A4.T14; row Gemma2-27B; condition DPI+IPI+MP; column Attack success rate |
| Gemma2-27B | Refusal rate | 0.50%% | DPI+IPI+MP#A4.T14; row Gemma2-27B; condition DPI+IPI+MP; column Refusal rate |
| LLaMA3-8B | Attack success rate | 55.75%% | DPI+IPI#A4.T14; row LLaMA3-8B; condition DPI+IPI; column Attack success rate |
| LLaMA3-8B | Refusal rate | 14.75%% | DPI+IPI#A4.T14; row LLaMA3-8B; condition DPI+IPI; column Refusal rate |
| LLaMA3-8B | Attack success rate | 37.75%% | DPI+MP#A4.T14; row LLaMA3-8B; condition DPI+MP; column Attack success rate |
| LLaMA3-8B | Refusal rate | 5.25%% | DPI+MP#A4.T14; row LLaMA3-8B; condition DPI+MP; column Refusal rate |
| LLaMA3-8B | Attack success rate | 4.25%% | IPI+MP#A4.T14; row LLaMA3-8B; condition IPI+MP; column Attack success rate |
| LLaMA3-8B | Refusal rate | 1.75%% | IPI+MP#A4.T14; row LLaMA3-8B; condition IPI+MP; column Refusal rate |
| LLaMA3-8B | Attack success rate | 40.75%% | DPI+IPI+MP#A4.T14; row LLaMA3-8B; condition DPI+IPI+MP; column Attack success rate |
| LLaMA3-8B | Refusal rate | 5.75%% | DPI+IPI+MP#A4.T14; row LLaMA3-8B; condition DPI+IPI+MP; column Refusal rate |
| LLaMA3-70B | Attack success rate | 86.50%% | DPI+IPI#A4.T14; row LLaMA3-70B; condition DPI+IPI; column Attack success rate |
| LLaMA3-70B | Refusal rate | 7.50%% | DPI+IPI#A4.T14; row LLaMA3-70B; condition DPI+IPI; column Refusal rate |
| LLaMA3-70B | Attack success rate | 83.75%% | DPI+MP#A4.T14; row LLaMA3-70B; condition DPI+MP; column Attack success rate |
| LLaMA3-70B | Refusal rate | 6.50%% | DPI+MP#A4.T14; row LLaMA3-70B; condition DPI+MP; column Refusal rate |
| LLaMA3-70B | Attack success rate | 27.50%% | IPI+MP#A4.T14; row LLaMA3-70B; condition IPI+MP; column Attack success rate |
| LLaMA3-70B | Refusal rate | 2.50%% | IPI+MP#A4.T14; row LLaMA3-70B; condition IPI+MP; column Refusal rate |
| LLaMA3-70B | Attack success rate | 85.50%% | DPI+IPI+MP#A4.T14; row LLaMA3-70B; condition DPI+IPI+MP; column Attack success rate |
| LLaMA3-70B | Refusal rate | 6.50%% | DPI+IPI+MP#A4.T14; row LLaMA3-70B; condition DPI+IPI+MP; column Refusal rate |
| LLaMA3.1-8B | Attack success rate | 68.25%% | DPI+IPI#A4.T14; row LLaMA3.1-8B; condition DPI+IPI; column Attack success rate |
| LLaMA3.1-8B | Refusal rate | 10.00%% | DPI+IPI#A4.T14; row LLaMA3.1-8B; condition DPI+IPI; column Refusal rate |
| LLaMA3.1-8B | Attack success rate | 67.50%% | DPI+MP#A4.T14; row LLaMA3.1-8B; condition DPI+MP; column Attack success rate |
| LLaMA3.1-8B | Refusal rate | 5.25%% | DPI+MP#A4.T14; row LLaMA3.1-8B; condition DPI+MP; column Refusal rate |
| LLaMA3.1-8B | Attack success rate | 6.25%% | IPI+MP#A4.T14; row LLaMA3.1-8B; condition IPI+MP; column Attack success rate |
| LLaMA3.1-8B | Refusal rate | 2.75%% | IPI+MP#A4.T14; row LLaMA3.1-8B; condition IPI+MP; column Refusal rate |
| LLaMA3.1-8B | Attack success rate | 73.50%% | DPI+IPI+MP#A4.T14; row LLaMA3.1-8B; condition DPI+IPI+MP; column Attack success rate |
| LLaMA3.1-8B | Refusal rate | 3.50%% | DPI+IPI+MP#A4.T14; row LLaMA3.1-8B; condition DPI+IPI+MP; column Refusal rate |
| LLaMA3.1-70B | Attack success rate | 88.00%% | DPI+IPI#A4.T14; row LLaMA3.1-70B; condition DPI+IPI; column Attack success rate |
| LLaMA3.1-70B | Refusal rate | 4.50%% | DPI+IPI#A4.T14; row LLaMA3.1-70B; condition DPI+IPI; column Refusal rate |
| LLaMA3.1-70B | Attack success rate | 95.75%% | DPI+MP#A4.T14; row LLaMA3.1-70B; condition DPI+MP; column Attack success rate |
| LLaMA3.1-70B | Refusal rate | 0.75%% | DPI+MP#A4.T14; row LLaMA3.1-70B; condition DPI+MP; column Refusal rate |
| LLaMA3.1-70B | Attack success rate | 10.25%% | IPI+MP#A4.T14; row LLaMA3.1-70B; condition IPI+MP; column Attack success rate |
| LLaMA3.1-70B | Refusal rate | 2.50%% | IPI+MP#A4.T14; row LLaMA3.1-70B; condition IPI+MP; column Refusal rate |
| LLaMA3.1-70B | Attack success rate | 94.50%% | DPI+IPI+MP#A4.T14; row LLaMA3.1-70B; condition DPI+IPI+MP; column Attack success rate |
| LLaMA3.1-70B | Refusal rate | 1.25%% | DPI+IPI+MP#A4.T14; row LLaMA3.1-70B; condition DPI+IPI+MP; column Refusal rate |
| Mixtral-8x7B | Attack success rate | 32.50%% | DPI+IPI#A4.T14; row Mixtral-8x7B; condition DPI+IPI; column Attack success rate |
| Mixtral-8x7B | Refusal rate | 11.25%% | DPI+IPI#A4.T14; row Mixtral-8x7B; condition DPI+IPI; column Refusal rate |
| Mixtral-8x7B | Attack success rate | 55.50%% | DPI+MP#A4.T14; row Mixtral-8x7B; condition DPI+MP; column Attack success rate |
| Mixtral-8x7B | Refusal rate | 7.75%% | DPI+MP#A4.T14; row Mixtral-8x7B; condition DPI+MP; column Refusal rate |
| Mixtral-8x7B | Attack success rate | 4.25%% | IPI+MP#A4.T14; row Mixtral-8x7B; condition IPI+MP; column Attack success rate |
| Mixtral-8x7B | Refusal rate | 5.50%% | IPI+MP#A4.T14; row Mixtral-8x7B; condition IPI+MP; column Refusal rate |
| Mixtral-8x7B | Attack success rate | 54.75%% | DPI+IPI+MP#A4.T14; row Mixtral-8x7B; condition DPI+IPI+MP; column Attack success rate |
| Mixtral-8x7B | Refusal rate | 6.75%% | DPI+IPI+MP#A4.T14; row Mixtral-8x7B; condition DPI+IPI+MP; column Refusal rate |
| Qwen2-7B | Attack success rate | 71.50%% | DPI+IPI#A4.T14; row Qwen2-7B; condition DPI+IPI; column Attack success rate |
| Qwen2-7B | Refusal rate | 6.50%% | DPI+IPI#A4.T14; row Qwen2-7B; condition DPI+IPI; column Refusal rate |
| Qwen2-7B | Attack success rate | 70.00%% | DPI+MP#A4.T14; row Qwen2-7B; condition DPI+MP; column Attack success rate |
| Qwen2-7B | Refusal rate | 1.50%% | DPI+MP#A4.T14; row Qwen2-7B; condition DPI+MP; column Refusal rate |
| Qwen2-7B | Attack success rate | 9.25%% | IPI+MP#A4.T14; row Qwen2-7B; condition IPI+MP; column Attack success rate |
| Qwen2-7B | Refusal rate | 7.00%% | IPI+MP#A4.T14; row Qwen2-7B; condition IPI+MP; column Refusal rate |
| Qwen2-7B | Attack success rate | 76.00%% | DPI+IPI+MP#A4.T14; row Qwen2-7B; condition DPI+IPI+MP; column Attack success rate |
| Qwen2-7B | Refusal rate | 2.50%% | DPI+IPI+MP#A4.T14; row Qwen2-7B; condition DPI+IPI+MP; column Refusal rate |
| Qwen2-72B | Attack success rate | 97.75%% | DPI+IPI#A4.T14; row Qwen2-72B; condition DPI+IPI; column Attack success rate |
| Qwen2-72B | Refusal rate | 1.75%% | DPI+IPI#A4.T14; row Qwen2-72B; condition DPI+IPI; column Refusal rate |
| Qwen2-72B | Attack success rate | 98.00%% | DPI+MP#A4.T14; row Qwen2-72B; condition DPI+MP; column Attack success rate |
| Qwen2-72B | Refusal rate | 3.00%% | DPI+MP#A4.T14; row Qwen2-72B; condition DPI+MP; column Refusal rate |
| Qwen2-72B | Attack success rate | 23.25%% | IPI+MP#A4.T14; row Qwen2-72B; condition IPI+MP; column Attack success rate |
| Qwen2-72B | Refusal rate | 1.75%% | IPI+MP#A4.T14; row Qwen2-72B; condition IPI+MP; column Refusal rate |
| Qwen2-72B | Attack success rate | 98.50%% | DPI+IPI+MP#A4.T14; row Qwen2-72B; condition DPI+IPI+MP; column Attack success rate |
| Qwen2-72B | Refusal rate | 0.75%% | DPI+IPI+MP#A4.T14; row Qwen2-72B; condition DPI+IPI+MP; column Refusal rate |
| Claude-3.5 Sonnet | Attack success rate | 88.00%% | DPI+IPI#A4.T14; row Claude-3.5 Sonnet; condition DPI+IPI; column Attack success rate |
| Claude-3.5 Sonnet | Refusal rate | 21.00%% | DPI+IPI#A4.T14; row Claude-3.5 Sonnet; condition DPI+IPI; column Refusal rate |
| Claude-3.5 Sonnet | Attack success rate | 94.50%% | DPI+MP#A4.T14; row Claude-3.5 Sonnet; condition DPI+MP; column Attack success rate |
| Claude-3.5 Sonnet | Refusal rate | 6.75%% | DPI+MP#A4.T14; row Claude-3.5 Sonnet; condition DPI+MP; column Refusal rate |
| Claude-3.5 Sonnet | Attack success rate | 58.25%% | IPI+MP#A4.T14; row Claude-3.5 Sonnet; condition IPI+MP; column Attack success rate |
| Claude-3.5 Sonnet | Refusal rate | 16.75%% | IPI+MP#A4.T14; row Claude-3.5 Sonnet; condition IPI+MP; column Refusal rate |
| Claude-3.5 Sonnet | Attack success rate | 94.50%% | DPI+IPI+MP#A4.T14; row Claude-3.5 Sonnet; condition DPI+IPI+MP; column Attack success rate |
| Claude-3.5 Sonnet | Refusal rate | 6.25%% | DPI+IPI+MP#A4.T14; row Claude-3.5 Sonnet; condition DPI+IPI+MP; column Refusal rate |
| GPT-3.5 Turbo | Attack success rate | 99.50%% | DPI+IPI#A4.T14; row GPT-3.5 Turbo; condition DPI+IPI; column Attack success rate |
| GPT-3.5 Turbo | Refusal rate | 0.00%% | DPI+IPI#A4.T14; row GPT-3.5 Turbo; condition DPI+IPI; column Refusal rate |
| GPT-3.5 Turbo | Attack success rate | 100.00%% | DPI+MP#A4.T14; row GPT-3.5 Turbo; condition DPI+MP; column Attack success rate |
| GPT-3.5 Turbo | Refusal rate | 0.25%% | DPI+MP#A4.T14; row GPT-3.5 Turbo; condition DPI+MP; column Refusal rate |
| GPT-3.5 Turbo | Attack success rate | 41.25%% | IPI+MP#A4.T14; row GPT-3.5 Turbo; condition IPI+MP; column Attack success rate |
| GPT-3.5 Turbo | Refusal rate | 12.75%% | IPI+MP#A4.T14; row GPT-3.5 Turbo; condition IPI+MP; column Refusal rate |
| GPT-3.5 Turbo | Attack success rate | 99.75%% | DPI+IPI+MP#A4.T14; row GPT-3.5 Turbo; condition DPI+IPI+MP; column Attack success rate |
| GPT-3.5 Turbo | Refusal rate | 0.00%% | DPI+IPI+MP#A4.T14; row GPT-3.5 Turbo; condition DPI+IPI+MP; column Refusal rate |
| GPT-4o | Attack success rate | 63.50%% | DPI+IPI#A4.T14; row GPT-4o; condition DPI+IPI; column Attack success rate |
| GPT-4o | Refusal rate | 19.25%% | DPI+IPI#A4.T14; row GPT-4o; condition DPI+IPI; column Refusal rate |
| GPT-4o | Attack success rate | 88.25%% | DPI+MP#A4.T14; row GPT-4o; condition DPI+MP; column Attack success rate |
| GPT-4o | Refusal rate | 7.50%% | DPI+MP#A4.T14; row GPT-4o; condition DPI+MP; column Refusal rate |
| GPT-4o | Attack success rate | 55.75%% | IPI+MP#A4.T14; row GPT-4o; condition IPI+MP; column Attack success rate |
| GPT-4o | Refusal rate | 0.75%% | IPI+MP#A4.T14; row GPT-4o; condition IPI+MP; column Refusal rate |
| GPT-4o | Attack success rate | 89.25%% | DPI+IPI+MP#A4.T14; row GPT-4o; condition DPI+IPI+MP; column Attack success rate |
| GPT-4o | Refusal rate | 5.50%% | DPI+IPI+MP#A4.T14; row GPT-4o; condition DPI+IPI+MP; column Refusal rate |
| GPT-4o-mini | Attack success rate | 88.50%% | DPI+IPI#A4.T14; row GPT-4o-mini; condition DPI+IPI; column Attack success rate |
| GPT-4o-mini | Refusal rate | 4.25%% | DPI+IPI#A4.T14; row GPT-4o-mini; condition DPI+IPI; column Refusal rate |
| GPT-4o-mini | Attack success rate | 97.50%% | DPI+MP#A4.T14; row GPT-4o-mini; condition DPI+MP; column Attack success rate |
| GPT-4o-mini | Refusal rate | 1.25%% | DPI+MP#A4.T14; row GPT-4o-mini; condition DPI+MP; column Refusal rate |
| GPT-4o-mini | Attack success rate | 38.50%% | IPI+MP#A4.T14; row GPT-4o-mini; condition IPI+MP; column Attack success rate |
| GPT-4o-mini | Refusal rate | 0.00%% | IPI+MP#A4.T14; row GPT-4o-mini; condition IPI+MP; column Refusal rate |
| GPT-4o-mini | Attack success rate | 96.75%% | DPI+IPI+MP#A4.T14; row GPT-4o-mini; condition DPI+IPI+MP; column Attack success rate |
| GPT-4o-mini | Refusal rate | 1.25%% | DPI+IPI+MP#A4.T14; row GPT-4o-mini; condition DPI+IPI+MP; column Refusal rate |
| Average | Attack success rate | 79.06%% | DPI+IPI#A4.T14; row Average; condition DPI+IPI; column Attack success rate |
| Average | Refusal rate | 8.04%% | DPI+IPI#A4.T14; row Average; condition DPI+IPI; column Refusal rate |
| Average | Attack success rate | 83.02%% | DPI+MP#A4.T14; row Average; condition DPI+MP; column Attack success rate |
| Average | Refusal rate | 3.73%% | DPI+MP#A4.T14; row Average; condition DPI+MP; column Refusal rate |
| Average | Attack success rate | 23.75%% | IPI+MP#A4.T14; row Average; condition IPI+MP; column Attack success rate |
| Average | Refusal rate | 6.46%% | IPI+MP#A4.T14; row Average; condition IPI+MP; column Refusal rate |
| Average | Attack success rate | 84.30%% | DPI+IPI+MP#A4.T14; row Average; condition DPI+IPI+MP; column Attack success rate |
| Average | Refusal rate | 3.22%% | DPI+IPI+MP#A4.T14; row Average; condition DPI+IPI+MP; column Refusal rate |
| Change versus table-specific baseline | Attack success rate | 6.37%percentage points | DPI+IPI#A4.T14; row Δ; condition DPI+IPI; column Attack success rate |
| Change versus table-specific baseline | Refusal rate | 1.50%percentage points | DPI+IPI#A4.T14; row Δ; condition DPI+IPI; column Refusal rate |
| Change versus table-specific baseline | Attack success rate | 10.33%percentage points | DPI+MP#A4.T14; row Δ; condition DPI+MP; column Attack success rate |
| Change versus table-specific baseline | Refusal rate | -2.80%percentage points | DPI+MP#A4.T14; row Δ; condition DPI+MP; column Refusal rate |
| Change versus table-specific baseline | Attack success rate | -3.80%percentage points | IPI+MP#A4.T14; row Δ; condition IPI+MP; column Attack success rate |
| Change versus table-specific baseline | Refusal rate | -2.15%percentage points | IPI+MP#A4.T14; row Δ; condition IPI+MP; column Refusal rate |
| Change versus table-specific baseline | Attack success rate | 11.62%percentage points | DPI+IPI+MP#A4.T14; row Δ; condition DPI+IPI+MP; column Attack success rate |
| Change versus table-specific baseline | Refusal rate | -3.32%percentage points | DPI+IPI+MP#A4.T14; row Δ; condition DPI+IPI+MP; column Refusal rate |
| Gemma2-9B | Benign performance | 21.00%% | PoT backdoor#A4.T16; row Gemma2-9B; condition PoT backdoor; column Benign performance |
| Gemma2-9B | Performance under no attack | 10.75%% | No attack#A4.T16; row Gemma2-9B; condition No attack; column Performance under no attack |
| Gemma2-27B | Benign performance | 37.75%% | PoT backdoor#A4.T16; row Gemma2-27B; condition PoT backdoor; column Benign performance |
| Gemma2-27B | Performance under no attack | 31.50%% | No attack#A4.T16; row Gemma2-27B; condition No attack; column Performance under no attack |
| LLaMA3-8B | Benign performance | 4.25%% | PoT backdoor#A4.T16; row LLaMA3-8B; condition PoT backdoor; column Benign performance |
| LLaMA3-8B | Performance under no attack | 1.50%% | No attack#A4.T16; row LLaMA3-8B; condition No attack; column Performance under no attack |
| LLaMA3-70B | Benign performance | 59.00%% | PoT backdoor#A4.T16; row LLaMA3-70B; condition PoT backdoor; column Benign performance |
| LLaMA3-70B | Performance under no attack | 66.50%% | No attack#A4.T16; row LLaMA3-70B; condition No attack; column Performance under no attack |
| LLaMA3.1-8B | Benign performance | 2.75%% | PoT backdoor#A4.T16; row LLaMA3.1-8B; condition PoT backdoor; column Benign performance |
| LLaMA3.1-8B | Performance under no attack | 0.75%% | No attack#A4.T16; row LLaMA3.1-8B; condition No attack; column Performance under no attack |
| LLaMA3.1-70B | Benign performance | 32.00%% | PoT backdoor#A4.T16; row LLaMA3.1-70B; condition PoT backdoor; column Benign performance |
| LLaMA3.1-70B | Performance under no attack | 21.25%% | No attack#A4.T16; row LLaMA3.1-70B; condition No attack; column Performance under no attack |
| Mixtral-8x7B | Benign performance | 1.00%% | PoT backdoor#A4.T16; row Mixtral-8x7B; condition PoT backdoor; column Benign performance |
| Mixtral-8x7B | Performance under no attack | 0.00%% | No attack#A4.T16; row Mixtral-8x7B; condition No attack; column Performance under no attack |
| Qwen2-7B | Benign performance | 8.50%% | PoT backdoor#A4.T16; row Qwen2-7B; condition PoT backdoor; column Benign performance |
| Qwen2-7B | Performance under no attack | 9.75%% | No attack#A4.T16; row Qwen2-7B; condition No attack; column Performance under no attack |
| Qwen2-72B | Benign performance | 2.25%% | PoT backdoor#A4.T16; row Qwen2-72B; condition PoT backdoor; column Benign performance |
| Qwen2-72B | Performance under no attack | 4.00%% | No attack#A4.T16; row Qwen2-72B; condition No attack; column Performance under no attack |
| Claude-3.5 Sonnet | Benign performance | 90.75%% | PoT backdoor#A4.T16; row Claude-3.5 Sonnet; condition PoT backdoor; column Benign performance |
| Claude-3.5 Sonnet | Performance under no attack | 100.00%% | No attack#A4.T16; row Claude-3.5 Sonnet; condition No attack; column Performance under no attack |
| GPT-3.5 Turbo | Benign performance | 17.25%% | PoT backdoor#A4.T16; row GPT-3.5 Turbo; condition PoT backdoor; column Benign performance |
| GPT-3.5 Turbo | Performance under no attack | 8.00%% | No attack#A4.T16; row GPT-3.5 Turbo; condition No attack; column Performance under no attack |
| GPT-4o | Benign performance | 77.75%% | PoT backdoor#A4.T16; row GPT-4o; condition PoT backdoor; column Benign performance |
| GPT-4o | Performance under no attack | 79.00%% | No attack#A4.T16; row GPT-4o; condition No attack; column Performance under no attack |
| GPT-4o-mini | Benign performance | 64.50%% | PoT backdoor#A4.T16; row GPT-4o-mini; condition PoT backdoor; column Benign performance |
| GPT-4o-mini | Performance under no attack | 50.00%% | No attack#A4.T16; row GPT-4o-mini; condition No attack; column Performance under no attack |
| Average | Benign performance | 32.21%% | PoT backdoor#A4.T16; row Average; condition PoT backdoor; column Benign performance |
| Average | Performance under no attack | 29.46%% | No attack#A4.T16; row Average; condition No attack; column Performance under no attack |
| Gemma2-9B | Performance under no attack | 10.75%% | No attack#A4.T19; row Gemma2-9B; condition No attack; column Performance under no attack |
| Gemma2-9B | Performance under no attack | 8.00%% | Delimiters#A4.T19; row Gemma2-9B; condition Delimiters; column Performance under no attack |
| Gemma2-9B | Change from no-defense PNA | -2.75%percentage points | Delimiters#A4.T19; row Gemma2-9B; condition Delimiters; column Change from no-defense PNA |
| Gemma2-9B | Performance under no attack | 9.00%% | Paraphrasing#A4.T19; row Gemma2-9B; condition Paraphrasing; column Performance under no attack |
| Gemma2-9B | Change from no-defense PNA | -1.75%percentage points | Paraphrasing#A4.T19; row Gemma2-9B; condition Paraphrasing; column Change from no-defense PNA |
| Gemma2-9B | Performance under no attack | 8.00%% | Instructional#A4.T19; row Gemma2-9B; condition Instructional; column Performance under no attack |
| Gemma2-9B | Change from no-defense PNA | -2.75%percentage points | Instructional#A4.T19; row Gemma2-9B; condition Instructional; column Change from no-defense PNA |
| Gemma2-9B | Performance under no attack | 14.00%% | Sandwich#A4.T19; row Gemma2-9B; condition Sandwich; column Performance under no attack |
| Gemma2-9B | Change from no-defense PNA | 3.25%percentage points | Sandwich#A4.T19; row Gemma2-9B; condition Sandwich; column Change from no-defense PNA |
| Gemma2-9B | Performance under no attack | 10.20%% | DPR#A4.T19; row Gemma2-9B; condition DPR; column Performance under no attack |
| Gemma2-9B | Change from no-defense PNA | -0.55%percentage points | DPR#A4.T19; row Gemma2-9B; condition DPR; column Change from no-defense PNA |
| Gemma2-27B | Performance under no attack | 31.50%% | No attack#A4.T19; row Gemma2-27B; condition No attack; column Performance under no attack |
| Gemma2-27B | Performance under no attack | 14.75%% | Delimiters#A4.T19; row Gemma2-27B; condition Delimiters; column Performance under no attack |
| Gemma2-27B | Change from no-defense PNA | -16.75%percentage points | Delimiters#A4.T19; row Gemma2-27B; condition Delimiters; column Change from no-defense PNA |
| Gemma2-27B | Performance under no attack | 24.75%% | Paraphrasing#A4.T19; row Gemma2-27B; condition Paraphrasing; column Performance under no attack |
| Gemma2-27B | Change from no-defense PNA | -6.75%percentage points | Paraphrasing#A4.T19; row Gemma2-27B; condition Paraphrasing; column Change from no-defense PNA |
| Gemma2-27B | Performance under no attack | 23.00%% | Instructional#A4.T19; row Gemma2-27B; condition Instructional; column Performance under no attack |
| Gemma2-27B | Change from no-defense PNA | -8.50%percentage points | Instructional#A4.T19; row Gemma2-27B; condition Instructional; column Change from no-defense PNA |
| Gemma2-27B | Performance under no attack | 27.00%% | Sandwich#A4.T19; row Gemma2-27B; condition Sandwich; column Performance under no attack |
| Gemma2-27B | Change from no-defense PNA | -4.50%percentage points | Sandwich#A4.T19; row Gemma2-27B; condition Sandwich; column Change from no-defense PNA |
| Gemma2-27B | Performance under no attack | 30.50%% | DPR#A4.T19; row Gemma2-27B; condition DPR; column Performance under no attack |
| Gemma2-27B | Change from no-defense PNA | -1.00%percentage points | DPR#A4.T19; row Gemma2-27B; condition DPR; column Change from no-defense PNA |
| LLaMA3-8B | Performance under no attack | 1.50%% | No attack#A4.T19; row LLaMA3-8B; condition No attack; column Performance under no attack |
| LLaMA3-8B | Performance under no attack | 1.75%% | Delimiters#A4.T19; row LLaMA3-8B; condition Delimiters; column Performance under no attack |
| LLaMA3-8B | Change from no-defense PNA | 0.25%percentage points | Delimiters#A4.T19; row LLaMA3-8B; condition Delimiters; column Change from no-defense PNA |
| LLaMA3-8B | Performance under no attack | 8.25%% | Paraphrasing#A4.T19; row LLaMA3-8B; condition Paraphrasing; column Performance under no attack |
| LLaMA3-8B | Change from no-defense PNA | 6.75%percentage points | Paraphrasing#A4.T19; row LLaMA3-8B; condition Paraphrasing; column Change from no-defense PNA |
| LLaMA3-8B | Performance under no attack | 5.00%% | Instructional#A4.T19; row LLaMA3-8B; condition Instructional; column Performance under no attack |
| LLaMA3-8B | Change from no-defense PNA | 3.50%percentage points | Instructional#A4.T19; row LLaMA3-8B; condition Instructional; column Change from no-defense PNA |
| LLaMA3-8B | Performance under no attack | 8.50%% | Sandwich#A4.T19; row LLaMA3-8B; condition Sandwich; column Performance under no attack |
| LLaMA3-8B | Change from no-defense PNA | 7.00%percentage points | Sandwich#A4.T19; row LLaMA3-8B; condition Sandwich; column Change from no-defense PNA |
| LLaMA3-8B | Performance under no attack | 7.50%% | DPR#A4.T19; row LLaMA3-8B; condition DPR; column Performance under no attack |
| LLaMA3-8B | Change from no-defense PNA | 6.00%percentage points | DPR#A4.T19; row LLaMA3-8B; condition DPR; column Change from no-defense PNA |
| LLaMA3-70B | Performance under no attack | 66.50%% | No attack#A4.T19; row LLaMA3-70B; condition No attack; column Performance under no attack |
| LLaMA3-70B | Performance under no attack | 58.50%% | Delimiters#A4.T19; row LLaMA3-70B; condition Delimiters; column Performance under no attack |
| LLaMA3-70B | Change from no-defense PNA | -8.00%percentage points | Delimiters#A4.T19; row LLaMA3-70B; condition Delimiters; column Change from no-defense PNA |
| LLaMA3-70B | Performance under no attack | 68.25%% | Paraphrasing#A4.T19; row LLaMA3-70B; condition Paraphrasing; column Performance under no attack |
| LLaMA3-70B | Change from no-defense PNA | 1.75%percentage points | Paraphrasing#A4.T19; row LLaMA3-70B; condition Paraphrasing; column Change from no-defense PNA |
| LLaMA3-70B | Performance under no attack | 70.00%% | Instructional#A4.T19; row LLaMA3-70B; condition Instructional; column Performance under no attack |
| LLaMA3-70B | Change from no-defense PNA | 3.50%percentage points | Instructional#A4.T19; row LLaMA3-70B; condition Instructional; column Change from no-defense PNA |
| LLaMA3-70B | Performance under no attack | 68.00%% | Sandwich#A4.T19; row LLaMA3-70B; condition Sandwich; column Performance under no attack |
| LLaMA3-70B | Change from no-defense PNA | 1.50%percentage points | Sandwich#A4.T19; row LLaMA3-70B; condition Sandwich; column Change from no-defense PNA |
| LLaMA3-70B | Performance under no attack | 64.50%% | DPR#A4.T19; row LLaMA3-70B; condition DPR; column Performance under no attack |
| LLaMA3-70B | Change from no-defense PNA | -2.00%percentage points | DPR#A4.T19; row LLaMA3-70B; condition DPR; column Change from no-defense PNA |
| LLaMA3.1-8B | Performance under no attack | 0.75%% | No attack#A4.T19; row LLaMA3.1-8B; condition No attack; column Performance under no attack |
| LLaMA3.1-8B | Performance under no attack | 0.00%% | Delimiters#A4.T19; row LLaMA3.1-8B; condition Delimiters; column Performance under no attack |
| LLaMA3.1-8B | Change from no-defense PNA | -0.75%percentage points | Delimiters#A4.T19; row LLaMA3.1-8B; condition Delimiters; column Change from no-defense PNA |
| LLaMA3.1-8B | Performance under no attack | 1.00%% | Paraphrasing#A4.T19; row LLaMA3.1-8B; condition Paraphrasing; column Performance under no attack |
| LLaMA3.1-8B | Change from no-defense PNA | 0.25%percentage points | Paraphrasing#A4.T19; row LLaMA3.1-8B; condition Paraphrasing; column Change from no-defense PNA |
| LLaMA3.1-8B | Performance under no attack | 0.75%% | Instructional#A4.T19; row LLaMA3.1-8B; condition Instructional; column Performance under no attack |
| LLaMA3.1-8B | Change from no-defense PNA | 0.00%percentage points | Instructional#A4.T19; row LLaMA3.1-8B; condition Instructional; column Change from no-defense PNA |
| LLaMA3.1-8B | Performance under no attack | 0.25%% | Sandwich#A4.T19; row LLaMA3.1-8B; condition Sandwich; column Performance under no attack |
| LLaMA3.1-8B | Change from no-defense PNA | -0.50%percentage points | Sandwich#A4.T19; row LLaMA3.1-8B; condition Sandwich; column Change from no-defense PNA |
| LLaMA3.1-8B | Performance under no attack | 0.50%% | DPR#A4.T19; row LLaMA3.1-8B; condition DPR; column Performance under no attack |
| LLaMA3.1-8B | Change from no-defense PNA | -0.25%percentage points | DPR#A4.T19; row LLaMA3.1-8B; condition DPR; column Change from no-defense PNA |
| LLaMA3.1-70B | Performance under no attack | 21.25%% | No attack#A4.T19; row LLaMA3.1-70B; condition No attack; column Performance under no attack |
| LLaMA3.1-70B | Performance under no attack | 9.50%% | Delimiters#A4.T19; row LLaMA3.1-70B; condition Delimiters; column Performance under no attack |
| LLaMA3.1-70B | Change from no-defense PNA | -11.75%percentage points | Delimiters#A4.T19; row LLaMA3.1-70B; condition Delimiters; column Change from no-defense PNA |
| LLaMA3.1-70B | Performance under no attack | 18.75%% | Paraphrasing#A4.T19; row LLaMA3.1-70B; condition Paraphrasing; column Performance under no attack |
| LLaMA3.1-70B | Change from no-defense PNA | -2.50%percentage points | Paraphrasing#A4.T19; row LLaMA3.1-70B; condition Paraphrasing; column Change from no-defense PNA |
| LLaMA3.1-70B | Performance under no attack | 14.50%% | Instructional#A4.T19; row LLaMA3.1-70B; condition Instructional; column Performance under no attack |
| LLaMA3.1-70B | Change from no-defense PNA | -6.75%percentage points | Instructional#A4.T19; row LLaMA3.1-70B; condition Instructional; column Change from no-defense PNA |
| LLaMA3.1-70B | Performance under no attack | 16.50%% | Sandwich#A4.T19; row LLaMA3.1-70B; condition Sandwich; column Performance under no attack |
| LLaMA3.1-70B | Change from no-defense PNA | -4.75%percentage points | Sandwich#A4.T19; row LLaMA3.1-70B; condition Sandwich; column Change from no-defense PNA |
| LLaMA3.1-70B | Performance under no attack | 13.80%% | DPR#A4.T19; row LLaMA3.1-70B; condition DPR; column Performance under no attack |
| LLaMA3.1-70B | Change from no-defense PNA | -7.45%percentage points | DPR#A4.T19; row LLaMA3.1-70B; condition DPR; column Change from no-defense PNA |
| Mixtral-8x7B | Performance under no attack | 0.00%% | No attack#A4.T19; row Mixtral-8x7B; condition No attack; column Performance under no attack |
| Mixtral-8x7B | Performance under no attack | 0.25%% | Delimiters#A4.T19; row Mixtral-8x7B; condition Delimiters; column Performance under no attack |
| Mixtral-8x7B | Change from no-defense PNA | 0.25%percentage points | Delimiters#A4.T19; row Mixtral-8x7B; condition Delimiters; column Change from no-defense PNA |
| Mixtral-8x7B | Performance under no attack | 0.00%% | Paraphrasing#A4.T19; row Mixtral-8x7B; condition Paraphrasing; column Performance under no attack |
| Mixtral-8x7B | Change from no-defense PNA | 0.00%percentage points | Paraphrasing#A4.T19; row Mixtral-8x7B; condition Paraphrasing; column Change from no-defense PNA |
| Mixtral-8x7B | Performance under no attack | 0.00%% | Instructional#A4.T19; row Mixtral-8x7B; condition Instructional; column Performance under no attack |
| Mixtral-8x7B | Change from no-defense PNA | 0.00%percentage points | Instructional#A4.T19; row Mixtral-8x7B; condition Instructional; column Change from no-defense PNA |
| Mixtral-8x7B | Performance under no attack | 0.25%% | Sandwich#A4.T19; row Mixtral-8x7B; condition Sandwich; column Performance under no attack |
| Mixtral-8x7B | Change from no-defense PNA | 0.25%percentage points | Sandwich#A4.T19; row Mixtral-8x7B; condition Sandwich; column Change from no-defense PNA |
| Mixtral-8x7B | Performance under no attack | 0.30%% | DPR#A4.T19; row Mixtral-8x7B; condition DPR; column Performance under no attack |
| Mixtral-8x7B | Change from no-defense PNA | 0.30%percentage points | DPR#A4.T19; row Mixtral-8x7B; condition DPR; column Change from no-defense PNA |
| Qwen2-7B | Performance under no attack | 9.75%% | No attack#A4.T19; row Qwen2-7B; condition No attack; column Performance under no attack |
| Qwen2-7B | Performance under no attack | 1.75%% | Delimiters#A4.T19; row Qwen2-7B; condition Delimiters; column Performance under no attack |
| Qwen2-7B | Change from no-defense PNA | -8.00%percentage points | Delimiters#A4.T19; row Qwen2-7B; condition Delimiters; column Change from no-defense PNA |
| Qwen2-7B | Performance under no attack | 7.00%% | Paraphrasing#A4.T19; row Qwen2-7B; condition Paraphrasing; column Performance under no attack |
| Qwen2-7B | Change from no-defense PNA | -2.75%percentage points | Paraphrasing#A4.T19; row Qwen2-7B; condition Paraphrasing; column Change from no-defense PNA |
| Qwen2-7B | Performance under no attack | 5.75%% | Instructional#A4.T19; row Qwen2-7B; condition Instructional; column Performance under no attack |
| Qwen2-7B | Change from no-defense PNA | -4.00%percentage points | Instructional#A4.T19; row Qwen2-7B; condition Instructional; column Change from no-defense PNA |
| Qwen2-7B | Performance under no attack | 5.25%% | Sandwich#A4.T19; row Qwen2-7B; condition Sandwich; column Performance under no attack |
| Qwen2-7B | Change from no-defense PNA | -4.50%percentage points | Sandwich#A4.T19; row Qwen2-7B; condition Sandwich; column Change from no-defense PNA |
| Qwen2-7B | Performance under no attack | 6.80%% | DPR#A4.T19; row Qwen2-7B; condition DPR; column Performance under no attack |
| Qwen2-7B | Change from no-defense PNA | -2.95%percentage points | DPR#A4.T19; row Qwen2-7B; condition DPR; column Change from no-defense PNA |
| Qwen2-72B | Performance under no attack | 4.00%% | No attack#A4.T19; row Qwen2-72B; condition No attack; column Performance under no attack |
| Qwen2-72B | Performance under no attack | 0.25%% | Delimiters#A4.T19; row Qwen2-72B; condition Delimiters; column Performance under no attack |
| Qwen2-72B | Change from no-defense PNA | -3.75%percentage points | Delimiters#A4.T19; row Qwen2-72B; condition Delimiters; column Change from no-defense PNA |
| Qwen2-72B | Performance under no attack | 0.00%% | Paraphrasing#A4.T19; row Qwen2-72B; condition Paraphrasing; column Performance under no attack |
| Qwen2-72B | Change from no-defense PNA | -4.00%percentage points | Paraphrasing#A4.T19; row Qwen2-72B; condition Paraphrasing; column Change from no-defense PNA |
| Qwen2-72B | Performance under no attack | 0.50%% | Instructional#A4.T19; row Qwen2-72B; condition Instructional; column Performance under no attack |
| Qwen2-72B | Change from no-defense PNA | -3.50%percentage points | Instructional#A4.T19; row Qwen2-72B; condition Instructional; column Change from no-defense PNA |
| Qwen2-72B | Performance under no attack | 0.50%% | Sandwich#A4.T19; row Qwen2-72B; condition Sandwich; column Performance under no attack |
| Qwen2-72B | Change from no-defense PNA | -3.50%percentage points | Sandwich#A4.T19; row Qwen2-72B; condition Sandwich; column Change from no-defense PNA |
| Qwen2-72B | Performance under no attack | 0.00%% | DPR#A4.T19; row Qwen2-72B; condition DPR; column Performance under no attack |
| Qwen2-72B | Change from no-defense PNA | -4.00%percentage points | DPR#A4.T19; row Qwen2-72B; condition DPR; column Change from no-defense PNA |
| Claude-3.5 Sonnet | Performance under no attack | 100.00%% | No attack#A4.T19; row Claude-3.5 Sonnet; condition No attack; column Performance under no attack |
| Claude-3.5 Sonnet | Performance under no attack | 90.00%% | Delimiters#A4.T19; row Claude-3.5 Sonnet; condition Delimiters; column Performance under no attack |
| Claude-3.5 Sonnet | Change from no-defense PNA | -10.00%percentage points | Delimiters#A4.T19; row Claude-3.5 Sonnet; condition Delimiters; column Change from no-defense PNA |
| Claude-3.5 Sonnet | Performance under no attack | 99.75%% | Paraphrasing#A4.T19; row Claude-3.5 Sonnet; condition Paraphrasing; column Performance under no attack |
| Claude-3.5 Sonnet | Change from no-defense PNA | -0.25%percentage points | Paraphrasing#A4.T19; row Claude-3.5 Sonnet; condition Paraphrasing; column Change from no-defense PNA |
| Claude-3.5 Sonnet | Performance under no attack | 90.00%% | Instructional#A4.T19; row Claude-3.5 Sonnet; condition Instructional; column Performance under no attack |
| Claude-3.5 Sonnet | Change from no-defense PNA | -10.00%percentage points | Instructional#A4.T19; row Claude-3.5 Sonnet; condition Instructional; column Change from no-defense PNA |
| Claude-3.5 Sonnet | Performance under no attack | 100.00%% | Sandwich#A4.T19; row Claude-3.5 Sonnet; condition Sandwich; column Performance under no attack |
| Claude-3.5 Sonnet | Change from no-defense PNA | 0.00%percentage points | Sandwich#A4.T19; row Claude-3.5 Sonnet; condition Sandwich; column Change from no-defense PNA |
| Claude-3.5 Sonnet | Performance under no attack | 95.80%% | DPR#A4.T19; row Claude-3.5 Sonnet; condition DPR; column Performance under no attack |
| Claude-3.5 Sonnet | Change from no-defense PNA | -4.20%percentage points | DPR#A4.T19; row Claude-3.5 Sonnet; condition DPR; column Change from no-defense PNA |
| GPT-3.5 Turbo | Performance under no attack | 8.00%% | No attack#A4.T19; row GPT-3.5 Turbo; condition No attack; column Performance under no attack |
| GPT-3.5 Turbo | Performance under no attack | 0.75%% | Delimiters#A4.T19; row GPT-3.5 Turbo; condition Delimiters; column Performance under no attack |
| GPT-3.5 Turbo | Change from no-defense PNA | -7.25%percentage points | Delimiters#A4.T19; row GPT-3.5 Turbo; condition Delimiters; column Change from no-defense PNA |
| GPT-3.5 Turbo | Performance under no attack | 6.75%% | Paraphrasing#A4.T19; row GPT-3.5 Turbo; condition Paraphrasing; column Performance under no attack |
| GPT-3.5 Turbo | Change from no-defense PNA | -1.25%percentage points | Paraphrasing#A4.T19; row GPT-3.5 Turbo; condition Paraphrasing; column Change from no-defense PNA |
| GPT-3.5 Turbo | Performance under no attack | 13.50%% | Instructional#A4.T19; row GPT-3.5 Turbo; condition Instructional; column Performance under no attack |
| GPT-3.5 Turbo | Change from no-defense PNA | 5.50%percentage points | Instructional#A4.T19; row GPT-3.5 Turbo; condition Instructional; column Change from no-defense PNA |
| GPT-3.5 Turbo | Performance under no attack | 10.25%% | Sandwich#A4.T19; row GPT-3.5 Turbo; condition Sandwich; column Performance under no attack |
| GPT-3.5 Turbo | Change from no-defense PNA | 2.25%percentage points | Sandwich#A4.T19; row GPT-3.5 Turbo; condition Sandwich; column Change from no-defense PNA |
| GPT-3.5 Turbo | Performance under no attack | 9.50%% | DPR#A4.T19; row GPT-3.5 Turbo; condition DPR; column Performance under no attack |
| GPT-3.5 Turbo | Change from no-defense PNA | 1.50%percentage points | DPR#A4.T19; row GPT-3.5 Turbo; condition DPR; column Change from no-defense PNA |
| GPT-4o | Performance under no attack | 79.00%% | No attack#A4.T19; row GPT-4o; condition No attack; column Performance under no attack |
| GPT-4o | Performance under no attack | 70.25%% | Delimiters#A4.T19; row GPT-4o; condition Delimiters; column Performance under no attack |
| GPT-4o | Change from no-defense PNA | -8.75%percentage points | Delimiters#A4.T19; row GPT-4o; condition Delimiters; column Change from no-defense PNA |
| GPT-4o | Performance under no attack | 80.00%% | Paraphrasing#A4.T19; row GPT-4o; condition Paraphrasing; column Performance under no attack |
| GPT-4o | Change from no-defense PNA | 1.00%percentage points | Paraphrasing#A4.T19; row GPT-4o; condition Paraphrasing; column Change from no-defense PNA |
| GPT-4o | Performance under no attack | 72.75%% | Instructional#A4.T19; row GPT-4o; condition Instructional; column Performance under no attack |
| GPT-4o | Change from no-defense PNA | -6.25%percentage points | Instructional#A4.T19; row GPT-4o; condition Instructional; column Change from no-defense PNA |
| GPT-4o | Performance under no attack | 79.50%% | Sandwich#A4.T19; row GPT-4o; condition Sandwich; column Performance under no attack |
| GPT-4o | Change from no-defense PNA | 0.50%percentage points | Sandwich#A4.T19; row GPT-4o; condition Sandwich; column Change from no-defense PNA |
| GPT-4o | Performance under no attack | 73.80%% | DPR#A4.T19; row GPT-4o; condition DPR; column Performance under no attack |
| GPT-4o | Change from no-defense PNA | -5.20%percentage points | DPR#A4.T19; row GPT-4o; condition DPR; column Change from no-defense PNA |
| GPT-4o-mini | Performance under no attack | 50.00%% | No attack#A4.T19; row GPT-4o-mini; condition No attack; column Performance under no attack |
| GPT-4o-mini | Performance under no attack | 37.00%% | Delimiters#A4.T19; row GPT-4o-mini; condition Delimiters; column Performance under no attack |
| GPT-4o-mini | Change from no-defense PNA | -13.00%percentage points | Delimiters#A4.T19; row GPT-4o-mini; condition Delimiters; column Change from no-defense PNA |
| GPT-4o-mini | Performance under no attack | 36.50%% | Paraphrasing#A4.T19; row GPT-4o-mini; condition Paraphrasing; column Performance under no attack |
| GPT-4o-mini | Change from no-defense PNA | -13.50%percentage points | Paraphrasing#A4.T19; row GPT-4o-mini; condition Paraphrasing; column Change from no-defense PNA |
| GPT-4o-mini | Performance under no attack | 42.50%% | Instructional#A4.T19; row GPT-4o-mini; condition Instructional; column Performance under no attack |
| GPT-4o-mini | Change from no-defense PNA | -7.50%percentage points | Instructional#A4.T19; row GPT-4o-mini; condition Instructional; column Change from no-defense PNA |
| GPT-4o-mini | Performance under no attack | 37.75%% | Sandwich#A4.T19; row GPT-4o-mini; condition Sandwich; column Performance under no attack |
| GPT-4o-mini | Change from no-defense PNA | -12.25%percentage points | Sandwich#A4.T19; row GPT-4o-mini; condition Sandwich; column Change from no-defense PNA |
| GPT-4o-mini | Performance under no attack | 29.70%% | DPR#A4.T19; row GPT-4o-mini; condition DPR; column Performance under no attack |
| GPT-4o-mini | Change from no-defense PNA | -20.30%percentage points | DPR#A4.T19; row GPT-4o-mini; condition DPR; column Change from no-defense PNA |
| Average | Performance under no attack | 29.46%% | No attack#A4.T19; row Average; condition No attack; column Performance under no attack |
| Average | Performance under no attack | 22.52%% | Delimiters#A4.T19; row Average; condition Delimiters; column Performance under no attack |
| Average | Change from no-defense PNA | -6.94%percentage points | Delimiters#A4.T19; row Average; condition Delimiters; column Change from no-defense PNA |
| Average | Performance under no attack | 27.69%% | Paraphrasing#A4.T19; row Average; condition Paraphrasing; column Performance under no attack |
| Average | Change from no-defense PNA | -1.77%percentage points | Paraphrasing#A4.T19; row Average; condition Paraphrasing; column Change from no-defense PNA |
| Average | Performance under no attack | 26.63%% | Instructional#A4.T19; row Average; condition Instructional; column Performance under no attack |
| Average | Change from no-defense PNA | -2.83%percentage points | Instructional#A4.T19; row Average; condition Instructional; column Change from no-defense PNA |
| Average | Performance under no attack | 28.29%% | Sandwich#A4.T19; row Average; condition Sandwich; column Performance under no attack |
| Average | Change from no-defense PNA | -1.17%percentage points | Sandwich#A4.T19; row Average; condition Sandwich; column Change from no-defense PNA |
| Average | Performance under no attack | 26.38%% | DPR#A4.T19; row Average; condition DPR; column Performance under no attack |
| Average | Change from no-defense PNA | -3.08%percentage points | DPR#A4.T19; row Average; condition DPR; column Change from no-defense PNA |
| Gemma2-9B | Attack success rate | 39.75%% | PoT baseline#A4.T20; row Gemma2-9B; condition PoT baseline; column Attack success rate |
| Gemma2-9B | Performance under no attack | 10.75%% | PoT baseline#A4.T20; row Gemma2-9B; condition PoT baseline; column Performance under no attack |
| Gemma2-9B | Attack success rate | 67.25%% | Shuffle defense#A4.T20; row Gemma2-9B; condition Shuffle defense; column Attack success rate |
| Gemma2-9B | Performance under no attack | 22.25%% | Shuffle defense#A4.T20; row Gemma2-9B; condition Shuffle defense; column Performance under no attack |
| Gemma2-9B | Attack success rate | 24.50%% | Paraphrase defense#A4.T20; row Gemma2-9B; condition Paraphrase defense; column Attack success rate |
| Gemma2-9B | Performance under no attack | 21.75%% | Paraphrase defense#A4.T20; row Gemma2-9B; condition Paraphrase defense; column Performance under no attack |
| Gemma2-27B | Attack success rate | 54.50%% | PoT baseline#A4.T20; row Gemma2-27B; condition PoT baseline; column Attack success rate |
| Gemma2-27B | Performance under no attack | 31.50%% | PoT baseline#A4.T20; row Gemma2-27B; condition PoT baseline; column Performance under no attack |
| Gemma2-27B | Attack success rate | 59.50%% | Shuffle defense#A4.T20; row Gemma2-27B; condition Shuffle defense; column Attack success rate |
| Gemma2-27B | Performance under no attack | 40.75%% | Shuffle defense#A4.T20; row Gemma2-27B; condition Shuffle defense; column Performance under no attack |
| Gemma2-27B | Attack success rate | 23.25%% | Paraphrase defense#A4.T20; row Gemma2-27B; condition Paraphrase defense; column Attack success rate |
| Gemma2-27B | Performance under no attack | 32.25%% | Paraphrase defense#A4.T20; row Gemma2-27B; condition Paraphrase defense; column Performance under no attack |
| LLaMA3-8B | Attack success rate | 21.50%% | PoT baseline#A4.T20; row LLaMA3-8B; condition PoT baseline; column Attack success rate |
| LLaMA3-8B | Performance under no attack | 1.50%% | PoT baseline#A4.T20; row LLaMA3-8B; condition PoT baseline; column Performance under no attack |
| LLaMA3-8B | Attack success rate | 2.25%% | Shuffle defense#A4.T20; row LLaMA3-8B; condition Shuffle defense; column Attack success rate |
| LLaMA3-8B | Performance under no attack | 3.50%% | Shuffle defense#A4.T20; row LLaMA3-8B; condition Shuffle defense; column Performance under no attack |
| LLaMA3-8B | Attack success rate | 5.00%% | Paraphrase defense#A4.T20; row LLaMA3-8B; condition Paraphrase defense; column Attack success rate |
| LLaMA3-8B | Performance under no attack | 6.00%% | Paraphrase defense#A4.T20; row LLaMA3-8B; condition Paraphrase defense; column Performance under no attack |
| LLaMA3-70B | Attack success rate | 57.00%% | PoT baseline#A4.T20; row LLaMA3-70B; condition PoT baseline; column Attack success rate |
| LLaMA3-70B | Performance under no attack | 66.50%% | PoT baseline#A4.T20; row LLaMA3-70B; condition PoT baseline; column Performance under no attack |
| LLaMA3-70B | Attack success rate | 63.75%% | Shuffle defense#A4.T20; row LLaMA3-70B; condition Shuffle defense; column Attack success rate |
| LLaMA3-70B | Performance under no attack | 54.50%% | Shuffle defense#A4.T20; row LLaMA3-70B; condition Shuffle defense; column Performance under no attack |
| LLaMA3-70B | Attack success rate | 44.75%% | Paraphrase defense#A4.T20; row LLaMA3-70B; condition Paraphrase defense; column Attack success rate |
| LLaMA3-70B | Performance under no attack | 52.75%% | Paraphrase defense#A4.T20; row LLaMA3-70B; condition Paraphrase defense; column Performance under no attack |
| LLaMA3.1-8B | Attack success rate | 19.00%% | PoT baseline#A4.T20; row LLaMA3.1-8B; condition PoT baseline; column Attack success rate |
| LLaMA3.1-8B | Performance under no attack | 0.75%% | PoT baseline#A4.T20; row LLaMA3.1-8B; condition PoT baseline; column Performance under no attack |
| LLaMA3.1-8B | Attack success rate | 17.25%% | Shuffle defense#A4.T20; row LLaMA3.1-8B; condition Shuffle defense; column Attack success rate |
| LLaMA3.1-8B | Performance under no attack | 2.75%% | Shuffle defense#A4.T20; row LLaMA3.1-8B; condition Shuffle defense; column Performance under no attack |
| LLaMA3.1-8B | Attack success rate | 17.50%% | Paraphrase defense#A4.T20; row LLaMA3.1-8B; condition Paraphrase defense; column Attack success rate |
| LLaMA3.1-8B | Performance under no attack | 2.50%% | Paraphrase defense#A4.T20; row LLaMA3.1-8B; condition Paraphrase defense; column Performance under no attack |
| LLaMA3.1-70B | Attack success rate | 59.75%% | PoT baseline#A4.T20; row LLaMA3.1-70B; condition PoT baseline; column Attack success rate |
| LLaMA3.1-70B | Performance under no attack | 21.25%% | PoT baseline#A4.T20; row LLaMA3.1-70B; condition PoT baseline; column Performance under no attack |
| LLaMA3.1-70B | Attack success rate | 69.00%% | Shuffle defense#A4.T20; row LLaMA3.1-70B; condition Shuffle defense; column Attack success rate |
| LLaMA3.1-70B | Performance under no attack | 43.00%% | Shuffle defense#A4.T20; row LLaMA3.1-70B; condition Shuffle defense; column Performance under no attack |
| LLaMA3.1-70B | Attack success rate | 42.00%% | Paraphrase defense#A4.T20; row LLaMA3.1-70B; condition Paraphrase defense; column Attack success rate |
| LLaMA3.1-70B | Performance under no attack | 30.00%% | Paraphrase defense#A4.T20; row LLaMA3.1-70B; condition Paraphrase defense; column Performance under no attack |
| Mixtral-8x7B | Attack success rate | 4.75%% | PoT baseline#A4.T20; row Mixtral-8x7B; condition PoT baseline; column Attack success rate |
| Mixtral-8x7B | Performance under no attack | 0.00%% | PoT baseline#A4.T20; row Mixtral-8x7B; condition PoT baseline; column Performance under no attack |
| Mixtral-8x7B | Attack success rate | 12.25%% | Shuffle defense#A4.T20; row Mixtral-8x7B; condition Shuffle defense; column Attack success rate |
| Mixtral-8x7B | Performance under no attack | 0.25%% | Shuffle defense#A4.T20; row Mixtral-8x7B; condition Shuffle defense; column Performance under no attack |
| Mixtral-8x7B | Attack success rate | 4.50%% | Paraphrase defense#A4.T20; row Mixtral-8x7B; condition Paraphrase defense; column Attack success rate |
| Mixtral-8x7B | Performance under no attack | 0.50%% | Paraphrase defense#A4.T20; row Mixtral-8x7B; condition Paraphrase defense; column Performance under no attack |
| Qwen2-7B | Attack success rate | 12.25%% | PoT baseline#A4.T20; row Qwen2-7B; condition PoT baseline; column Attack success rate |
| Qwen2-7B | Performance under no attack | 9.75%% | PoT baseline#A4.T20; row Qwen2-7B; condition PoT baseline; column Performance under no attack |
| Qwen2-7B | Attack success rate | 14.50%% | Shuffle defense#A4.T20; row Qwen2-7B; condition Shuffle defense; column Attack success rate |
| Qwen2-7B | Performance under no attack | 13.00%% | Shuffle defense#A4.T20; row Qwen2-7B; condition Shuffle defense; column Performance under no attack |
| Qwen2-7B | Attack success rate | 11.00%% | Paraphrase defense#A4.T20; row Qwen2-7B; condition Paraphrase defense; column Attack success rate |
| Qwen2-7B | Performance under no attack | 10.25%% | Paraphrase defense#A4.T20; row Qwen2-7B; condition Paraphrase defense; column Performance under no attack |
| Qwen2-72B | Attack success rate | 57.75%% | PoT baseline#A4.T20; row Qwen2-72B; condition PoT baseline; column Attack success rate |
| Qwen2-72B | Performance under no attack | 4.00%% | PoT baseline#A4.T20; row Qwen2-72B; condition PoT baseline; column Performance under no attack |
| Qwen2-72B | Attack success rate | 22.75%% | Shuffle defense#A4.T20; row Qwen2-72B; condition Shuffle defense; column Attack success rate |
| Qwen2-72B | Performance under no attack | 10.75%% | Shuffle defense#A4.T20; row Qwen2-72B; condition Shuffle defense; column Performance under no attack |
| Qwen2-72B | Attack success rate | 37.75%% | Paraphrase defense#A4.T20; row Qwen2-72B; condition Paraphrase defense; column Attack success rate |
| Qwen2-72B | Performance under no attack | 18.00%% | Paraphrase defense#A4.T20; row Qwen2-72B; condition Paraphrase defense; column Performance under no attack |
| Claude-3.5 Sonnet | Attack success rate | 17.50%% | PoT baseline#A4.T20; row Claude-3.5 Sonnet; condition PoT baseline; column Attack success rate |
| Claude-3.5 Sonnet | Performance under no attack | 100.00%% | PoT baseline#A4.T20; row Claude-3.5 Sonnet; condition PoT baseline; column Performance under no attack |
| Claude-3.5 Sonnet | Attack success rate | 93.50%% | Shuffle defense#A4.T20; row Claude-3.5 Sonnet; condition Shuffle defense; column Attack success rate |
| Claude-3.5 Sonnet | Performance under no attack | 81.50%% | Shuffle defense#A4.T20; row Claude-3.5 Sonnet; condition Shuffle defense; column Performance under no attack |
| Claude-3.5 Sonnet | Attack success rate | 13.75%% | Paraphrase defense#A4.T20; row Claude-3.5 Sonnet; condition Paraphrase defense; column Attack success rate |
| Claude-3.5 Sonnet | Performance under no attack | 82.75%% | Paraphrase defense#A4.T20; row Claude-3.5 Sonnet; condition Paraphrase defense; column Performance under no attack |
| GPT-3.5 Turbo | Attack success rate | 8.25%% | PoT baseline#A4.T20; row GPT-3.5 Turbo; condition PoT baseline; column Attack success rate |
| GPT-3.5 Turbo | Performance under no attack | 8.00%% | PoT baseline#A4.T20; row GPT-3.5 Turbo; condition PoT baseline; column Performance under no attack |
| GPT-3.5 Turbo | Attack success rate | 16.50%% | Shuffle defense#A4.T20; row GPT-3.5 Turbo; condition Shuffle defense; column Attack success rate |
| GPT-3.5 Turbo | Performance under no attack | 16.75%% | Shuffle defense#A4.T20; row GPT-3.5 Turbo; condition Shuffle defense; column Performance under no attack |
| GPT-3.5 Turbo | Attack success rate | 6.25%% | Paraphrase defense#A4.T20; row GPT-3.5 Turbo; condition Paraphrase defense; column Attack success rate |
| GPT-3.5 Turbo | Performance under no attack | 23.50%% | Paraphrase defense#A4.T20; row GPT-3.5 Turbo; condition Paraphrase defense; column Performance under no attack |
| GPT-4o | Attack success rate | 100.00%% | PoT baseline#A4.T20; row GPT-4o; condition PoT baseline; column Attack success rate |
| GPT-4o | Performance under no attack | 79.00%% | PoT baseline#A4.T20; row GPT-4o; condition PoT baseline; column Performance under no attack |
| GPT-4o | Attack success rate | 98.50%% | Shuffle defense#A4.T20; row GPT-4o; condition Shuffle defense; column Attack success rate |
| GPT-4o | Performance under no attack | 78.50%% | Shuffle defense#A4.T20; row GPT-4o; condition Shuffle defense; column Performance under no attack |
| GPT-4o | Attack success rate | 84.75%% | Paraphrase defense#A4.T20; row GPT-4o; condition Paraphrase defense; column Attack success rate |
| GPT-4o | Performance under no attack | 88.00%% | Paraphrase defense#A4.T20; row GPT-4o; condition Paraphrase defense; column Performance under no attack |
| GPT-4o-mini | Attack success rate | 95.50%% | PoT baseline#A4.T20; row GPT-4o-mini; condition PoT baseline; column Attack success rate |
| GPT-4o-mini | Performance under no attack | 50.00%% | PoT baseline#A4.T20; row GPT-4o-mini; condition PoT baseline; column Performance under no attack |
| GPT-4o-mini | Attack success rate | 39.75%% | Shuffle defense#A4.T20; row GPT-4o-mini; condition Shuffle defense; column Attack success rate |
| GPT-4o-mini | Performance under no attack | 63.75%% | Shuffle defense#A4.T20; row GPT-4o-mini; condition Shuffle defense; column Performance under no attack |
| GPT-4o-mini | Attack success rate | 62.75%% | Paraphrase defense#A4.T20; row GPT-4o-mini; condition Paraphrase defense; column Attack success rate |
| GPT-4o-mini | Performance under no attack | 79.00%% | Paraphrase defense#A4.T20; row GPT-4o-mini; condition Paraphrase defense; column Performance under no attack |
| Average | Attack success rate | 42.12%% | PoT baseline#A4.T20; row Average; condition PoT baseline; column Attack success rate |
| Average | Performance under no attack | 29.46%% | PoT baseline#A4.T20; row Average; condition PoT baseline; column Performance under no attack |
| Average | Attack success rate | 44.37%% | Shuffle defense#A4.T20; row Average; condition Shuffle defense; column Attack success rate |
| Average | Performance under no attack | 33.17%% | Shuffle defense#A4.T20; row Average; condition Shuffle defense; column Performance under no attack |
| Average | Attack success rate | 29.06%% | Paraphrase defense#A4.T20; row Average; condition Paraphrase defense; column Attack success rate |
| Average | Performance under no attack | 34.40%% | Paraphrase defense#A4.T20; row Average; condition Paraphrase defense; column Performance under no attack |
| Gemma2-9B | False negative rate | 0.658ratio | Memory attack detection#A4.T21; row Gemma2-9B; condition Memory attack detection; column False negative rate |
| Gemma2-9B | False positive rate | 0.204ratio | Memory attack detection#A4.T21; row Gemma2-9B; condition Memory attack detection; column False positive rate |
| Gemma2-27B | False negative rate | 0.655ratio | Memory attack detection#A4.T21; row Gemma2-27B; condition Memory attack detection; column False negative rate |
| Gemma2-27B | False positive rate | 0.201ratio | Memory attack detection#A4.T21; row Gemma2-27B; condition Memory attack detection; column False positive rate |
| LLaMA3-8B | False negative rate | 0.654ratio | Memory attack detection#A4.T21; row LLaMA3-8B; condition Memory attack detection; column False negative rate |
| LLaMA3-8B | False positive rate | 0.204ratio | Memory attack detection#A4.T21; row LLaMA3-8B; condition Memory attack detection; column False positive rate |
| LLaMA3-70B | False negative rate | 0.661ratio | Memory attack detection#A4.T21; row LLaMA3-70B; condition Memory attack detection; column False negative rate |
| LLaMA3-70B | False positive rate | 0.202ratio | Memory attack detection#A4.T21; row LLaMA3-70B; condition Memory attack detection; column False positive rate |
| LLaMA3.1-8B | False negative rate | 0.656ratio | Memory attack detection#A4.T21; row LLaMA3.1-8B; condition Memory attack detection; column False negative rate |
| LLaMA3.1-8B | False positive rate | 0.200ratio | Memory attack detection#A4.T21; row LLaMA3.1-8B; condition Memory attack detection; column False positive rate |
| LLaMA3.1-70B | False negative rate | 0.659ratio | Memory attack detection#A4.T21; row LLaMA3.1-70B; condition Memory attack detection; column False negative rate |
| LLaMA3.1-70B | False positive rate | 0.197ratio | Memory attack detection#A4.T21; row LLaMA3.1-70B; condition Memory attack detection; column False positive rate |
| Mixtral-8x7B | False negative rate | 0.665ratio | Memory attack detection#A4.T21; row Mixtral-8x7B; condition Memory attack detection; column False negative rate |
| Mixtral-8x7B | False positive rate | 0.203ratio | Memory attack detection#A4.T21; row Mixtral-8x7B; condition Memory attack detection; column False positive rate |
| Qwen2-7B | False negative rate | 0.657ratio | Memory attack detection#A4.T21; row Qwen2-7B; condition Memory attack detection; column False negative rate |
| Qwen2-7B | False positive rate | 0.193ratio | Memory attack detection#A4.T21; row Qwen2-7B; condition Memory attack detection; column False positive rate |
| Qwen2-72B | False negative rate | 0.671ratio | Memory attack detection#A4.T21; row Qwen2-72B; condition Memory attack detection; column False negative rate |
| Qwen2-72B | False positive rate | 0.198ratio | Memory attack detection#A4.T21; row Qwen2-72B; condition Memory attack detection; column False positive rate |
| Claude-3.5 Sonnet | False negative rate | 0.663ratio | Memory attack detection#A4.T21; row Claude-3.5 Sonnet; condition Memory attack detection; column False negative rate |
| Claude-3.5 Sonnet | False positive rate | 0.199ratio | Memory attack detection#A4.T21; row Claude-3.5 Sonnet; condition Memory attack detection; column False positive rate |
| GPT-3.5 Turbo | False negative rate | 0.661ratio | Memory attack detection#A4.T21; row GPT-3.5 Turbo; condition Memory attack detection; column False negative rate |
| GPT-3.5 Turbo | False positive rate | 0.198ratio | Memory attack detection#A4.T21; row GPT-3.5 Turbo; condition Memory attack detection; column False positive rate |
| GPT-4o | False negative rate | 0.664ratio | Memory attack detection#A4.T21; row GPT-4o; condition Memory attack detection; column False negative rate |
| GPT-4o | False positive rate | 0.203ratio | Memory attack detection#A4.T21; row GPT-4o; condition Memory attack detection; column False positive rate |
| GPT-4o-mini | False negative rate | 0.657ratio | Memory attack detection#A4.T21; row GPT-4o-mini; condition Memory attack detection; column False negative rate |
| GPT-4o-mini | False positive rate | 0.200ratio | Memory attack detection#A4.T21; row GPT-4o-mini; condition Memory attack detection; column False positive rate |
| Average | False negative rate | 0.660ratio | Memory attack detection#A4.T21; row Average; condition Memory attack detection; column False negative rate |
| Average | False positive rate | 0.200ratio | Memory attack detection#A4.T21; row Average; condition Memory attack detection; column False positive rate |
| Aggregate across evaluated backbones | Attack success rate | 78.38%% | Combined Attack / DPI#A4.T17; row Aggregate across evaluated backbones; condition Combined Attack / DPI; column Attack success rate |
| Aggregate across evaluated backbones | Refusal rate | 5.35%% | Combined Attack / DPI#A4.T17; row Aggregate across evaluated backbones; condition Combined Attack / DPI; column Refusal rate |
| Aggregate across evaluated backbones | Attack success rate | 27.98%% | Combined Attack / IPI#A4.T17; row Aggregate across evaluated backbones; condition Combined Attack / IPI; column Attack success rate |
| Aggregate across evaluated backbones | Refusal rate | 10.27%% | Combined Attack / IPI#A4.T17; row Aggregate across evaluated backbones; condition Combined Attack / IPI; column Refusal rate |
| Aggregate across evaluated backbones | Attack success rate | 7.65%% | Combined Attack / Memory poisoning#A4.T17; row Aggregate across evaluated backbones; condition Combined Attack / Memory poisoning; column Attack success rate |
| Aggregate across evaluated backbones | Refusal rate | 4.75%% | Combined Attack / Memory poisoning#A4.T17; row Aggregate across evaluated backbones; condition Combined Attack / Memory poisoning; column Refusal rate |
| Aggregate across evaluated backbones | Attack success rate | 38.01%% | Combined Attack / Average#A4.T17; row Aggregate across evaluated backbones; condition Combined Attack / Average; column Attack success rate |
| Aggregate across evaluated backbones | Refusal rate | 6.79%% | Combined Attack / Average#A4.T17; row Aggregate across evaluated backbones; condition Combined Attack / Average; column Refusal rate |
| Aggregate across evaluated backbones | Attack success rate | 73.85%% | Context Ignoring / DPI#A4.T17; row Aggregate across evaluated backbones; condition Context Ignoring / DPI; column Attack success rate |
| Aggregate across evaluated backbones | Refusal rate | 6.33%% | Context Ignoring / DPI#A4.T17; row Aggregate across evaluated backbones; condition Context Ignoring / DPI; column Refusal rate |
| Aggregate across evaluated backbones | Attack success rate | 27.29%% | Context Ignoring / IPI#A4.T17; row Aggregate across evaluated backbones; condition Context Ignoring / IPI; column Attack success rate |
| Aggregate across evaluated backbones | Refusal rate | 9.50%% | Context Ignoring / IPI#A4.T17; row Aggregate across evaluated backbones; condition Context Ignoring / IPI; column Refusal rate |
| Aggregate across evaluated backbones | Attack success rate | 8.52%% | Context Ignoring / Memory poisoning#A4.T17; row Aggregate across evaluated backbones; condition Context Ignoring / Memory poisoning; column Attack success rate |
| Aggregate across evaluated backbones | Refusal rate | 4.58%% | Context Ignoring / Memory poisoning#A4.T17; row Aggregate across evaluated backbones; condition Context Ignoring / Memory poisoning; column Refusal rate |
| Aggregate across evaluated backbones | Attack success rate | 36.55%% | Context Ignoring / Average#A4.T17; row Aggregate across evaluated backbones; condition Context Ignoring / Average; column Attack success rate |
| Aggregate across evaluated backbones | Refusal rate | 6.80%% | Context Ignoring / Average#A4.T17; row Aggregate across evaluated backbones; condition Context Ignoring / Average; column Refusal rate |
| Aggregate across evaluated backbones | Attack success rate | 68.73%% | Escape Characters / DPI#A4.T17; row Aggregate across evaluated backbones; condition Escape Characters / DPI; column Attack success rate |
| Aggregate across evaluated backbones | Refusal rate | 7.21%% | Escape Characters / DPI#A4.T17; row Aggregate across evaluated backbones; condition Escape Characters / DPI; column Refusal rate |
| Aggregate across evaluated backbones | Attack success rate | 27.81%% | Escape Characters / IPI#A4.T17; row Aggregate across evaluated backbones; condition Escape Characters / IPI; column Attack success rate |
| Aggregate across evaluated backbones | Refusal rate | 7.62%% | Escape Characters / IPI#A4.T17; row Aggregate across evaluated backbones; condition Escape Characters / IPI; column Refusal rate |
| Aggregate across evaluated backbones | Attack success rate | 7.71%% | Escape Characters / Memory poisoning#A4.T17; row Aggregate across evaluated backbones; condition Escape Characters / Memory poisoning; column Attack success rate |
| Aggregate across evaluated backbones | Refusal rate | 4.38%% | Escape Characters / Memory poisoning#A4.T17; row Aggregate across evaluated backbones; condition Escape Characters / Memory poisoning; column Refusal rate |
| Aggregate across evaluated backbones | Attack success rate | 34.75%% | Escape Characters / Average#A4.T17; row Aggregate across evaluated backbones; condition Escape Characters / Average; column Attack success rate |
| Aggregate across evaluated backbones | Refusal rate | 6.40%% | Escape Characters / Average#A4.T17; row Aggregate across evaluated backbones; condition Escape Characters / Average; column Refusal rate |
| Aggregate across evaluated backbones | Attack success rate | 71.94%% | Fake Completion / DPI#A4.T17; row Aggregate across evaluated backbones; condition Fake Completion / DPI; column Attack success rate |
| Aggregate across evaluated backbones | Refusal rate | 5.63%% | Fake Completion / DPI#A4.T17; row Aggregate across evaluated backbones; condition Fake Completion / DPI; column Refusal rate |
| Aggregate across evaluated backbones | Attack success rate | 26.62%% | Fake Completion / IPI#A4.T17; row Aggregate across evaluated backbones; condition Fake Completion / IPI; column Attack success rate |
| Aggregate across evaluated backbones | Refusal rate | 8.19%% | Fake Completion / IPI#A4.T17; row Aggregate across evaluated backbones; condition Fake Completion / IPI; column Refusal rate |
| Aggregate across evaluated backbones | Attack success rate | 8.00%% | Fake Completion / Memory poisoning#A4.T17; row Aggregate across evaluated backbones; condition Fake Completion / Memory poisoning; column Attack success rate |
| Aggregate across evaluated backbones | Refusal rate | 4.87%% | Fake Completion / Memory poisoning#A4.T17; row Aggregate across evaluated backbones; condition Fake Completion / Memory poisoning; column Refusal rate |
| Aggregate across evaluated backbones | Attack success rate | 35.52%% | Fake Completion / Average#A4.T17; row Aggregate across evaluated backbones; condition Fake Completion / Average; column Attack success rate |
| Aggregate across evaluated backbones | Refusal rate | 6.23%% | Fake Completion / Average#A4.T17; row Aggregate across evaluated backbones; condition Fake Completion / Average; column Refusal rate |
| Aggregate across evaluated backbones | Attack success rate | 70.52%% | Naive / DPI#A4.T17; row Aggregate across evaluated backbones; condition Naive / DPI; column Attack success rate |
| Aggregate across evaluated backbones | Refusal rate | 8.15%% | Naive / DPI#A4.T17; row Aggregate across evaluated backbones; condition Naive / DPI; column Refusal rate |
| Aggregate across evaluated backbones | Attack success rate | 28.04%% | Naive / IPI#A4.T17; row Aggregate across evaluated backbones; condition Naive / IPI; column Attack success rate |
| Aggregate across evaluated backbones | Refusal rate | 7.46%% | Naive / IPI#A4.T17; row Aggregate across evaluated backbones; condition Naive / IPI; column Refusal rate |
| Aggregate across evaluated backbones | Attack success rate | 7.73%% | Naive / Memory poisoning#A4.T17; row Aggregate across evaluated backbones; condition Naive / Memory poisoning; column Attack success rate |
| Aggregate across evaluated backbones | Refusal rate | 4.56%% | Naive / Memory poisoning#A4.T17; row Aggregate across evaluated backbones; condition Naive / Memory poisoning; column Refusal rate |
| Aggregate across evaluated backbones | Attack success rate | 35.43%% | Naive / Average#A4.T17; row Aggregate across evaluated backbones; condition Naive / Average; column Attack success rate |
| Aggregate across evaluated backbones | Refusal rate | 6.72%% | Naive / Average#A4.T17; row Aggregate across evaluated backbones; condition Naive / Average; column Refusal rate |
| Aggregate across evaluated backbones | Attack success rate | 72.68%% | Average / DPI#A4.T17; row Aggregate across evaluated backbones; condition Average / DPI; column Attack success rate |
| Aggregate across evaluated backbones | Refusal rate | 6.53%% | Average / DPI#A4.T17; row Aggregate across evaluated backbones; condition Average / DPI; column Refusal rate |
| Aggregate across evaluated backbones | Attack success rate | 27.55%% | Average / IPI#A4.T17; row Aggregate across evaluated backbones; condition Average / IPI; column Attack success rate |
| Aggregate across evaluated backbones | Refusal rate | 8.61%% | Average / IPI#A4.T17; row Aggregate across evaluated backbones; condition Average / IPI; column Refusal rate |
| Aggregate across evaluated backbones | Attack success rate | 7.92%% | Average / Memory poisoning#A4.T17; row Aggregate across evaluated backbones; condition Average / Memory poisoning; column Attack success rate |
| Aggregate across evaluated backbones | Refusal rate | 4.63%% | Average / Memory poisoning#A4.T17; row Aggregate across evaluated backbones; condition Average / Memory poisoning; column Refusal rate |
| Aggregate across evaluated backbones | Attack success rate | 36.05%% | Average / Average#A4.T17; row Aggregate across evaluated backbones; condition Average / Average; column Attack success rate |
| Aggregate across evaluated backbones | Refusal rate | 6.59%% | Average / Average#A4.T17; row Aggregate across evaluated backbones; condition Average / Average; column Refusal rate |
| Aggregate across evaluated backbones | Attack success rate | 74.59%% | No / DPI#A4.T18; row Aggregate across evaluated backbones; condition No / DPI; column Attack success rate |
| Aggregate across evaluated backbones | Refusal rate | 3.51%% | No / DPI#A4.T18; row Aggregate across evaluated backbones; condition No / DPI; column Refusal rate |
| Aggregate across evaluated backbones | Attack success rate | 32.67%% | No / IPI#A4.T18; row Aggregate across evaluated backbones; condition No / IPI; column Attack success rate |
| Aggregate across evaluated backbones | Refusal rate | 6.52%% | No / IPI#A4.T18; row Aggregate across evaluated backbones; condition No / IPI; column Refusal rate |
| Aggregate across evaluated backbones | Attack success rate | 9.68%% | No / Memory poisoning#A4.T18; row Aggregate across evaluated backbones; condition No / Memory poisoning; column Attack success rate |
| Aggregate across evaluated backbones | Refusal rate | 4.57%% | No / Memory poisoning#A4.T18; row Aggregate across evaluated backbones; condition No / Memory poisoning; column Refusal rate |
| Aggregate across evaluated backbones | Attack success rate | 38.98%% | No / Average#A4.T18; row Aggregate across evaluated backbones; condition No / Average; column Attack success rate |
| Aggregate across evaluated backbones | Refusal rate | 4.87%% | No / Average#A4.T18; row Aggregate across evaluated backbones; condition No / Average; column Refusal rate |
| Aggregate across evaluated backbones | Attack success rate | 70.78%% | Yes / DPI#A4.T18; row Aggregate across evaluated backbones; condition Yes / DPI; column Attack success rate |
| Aggregate across evaluated backbones | Refusal rate | 9.56%% | Yes / DPI#A4.T18; row Aggregate across evaluated backbones; condition Yes / DPI; column Refusal rate |
| Aggregate across evaluated backbones | Attack success rate | 22.42%% | Yes / IPI#A4.T18; row Aggregate across evaluated backbones; condition Yes / IPI; column Attack success rate |
| Aggregate across evaluated backbones | Refusal rate | 10.69%% | Yes / IPI#A4.T18; row Aggregate across evaluated backbones; condition Yes / IPI; column Refusal rate |
| Aggregate across evaluated backbones | Attack success rate | 6.16%% | Yes / Memory poisoning#A4.T18; row Aggregate across evaluated backbones; condition Yes / Memory poisoning; column Attack success rate |
| Aggregate across evaluated backbones | Refusal rate | 4.68%% | Yes / Memory poisoning#A4.T18; row Aggregate across evaluated backbones; condition Yes / Memory poisoning; column Refusal rate |
| Aggregate across evaluated backbones | Attack success rate | 33.12%% | Yes / Average#A4.T18; row Aggregate across evaluated backbones; condition Yes / Average; column Attack success rate |
| Aggregate across evaluated backbones | Refusal rate | 8.31%% | Yes / Average#A4.T18; row Aggregate across evaluated backbones; condition Yes / Average; column Refusal rate |
| Aggregate across evaluated backbones | Attack success rate | 72.68%% | Average / DPI#A4.T18; row Aggregate across evaluated backbones; condition Average / DPI; column Attack success rate |
| Aggregate across evaluated backbones | Refusal rate | 6.53%% | Average / DPI#A4.T18; row Aggregate across evaluated backbones; condition Average / DPI; column Refusal rate |
| Aggregate across evaluated backbones | Attack success rate | 27.55%% | Average / IPI#A4.T18; row Aggregate across evaluated backbones; condition Average / IPI; column Attack success rate |
| Aggregate across evaluated backbones | Refusal rate | 8.61%% | Average / IPI#A4.T18; row Aggregate across evaluated backbones; condition Average / IPI; column Refusal rate |
| Aggregate across evaluated backbones | Attack success rate | 7.92%% | Average / Memory poisoning#A4.T18; row Aggregate across evaluated backbones; condition Average / Memory poisoning; column Attack success rate |
| Aggregate across evaluated backbones | Refusal rate | 4.63%% | Average / Memory poisoning#A4.T18; row Aggregate across evaluated backbones; condition Average / Memory poisoning; column Refusal rate |
| Aggregate across evaluated backbones | Attack success rate | 36.05%% | Average / Average#A4.T18; row Aggregate across evaluated backbones; condition Average / Average; column Attack success rate |
| Aggregate across evaluated backbones | Refusal rate | 6.59%% | Average / Average#A4.T18; row Aggregate across evaluated backbones; condition Average / Average; column Refusal rate |
Reading scope & review
Selected original HTML sections and table context; not a complete paper/appendix audit or independent reproduction.
Model-assisted reading; independent evidence review pending.
Check the original source ↗