NUWA RESEARCH RECORD
Agent Security Bench (ASB): Formalizing and Benchmarking Attacks and Defenses in LLM-based Agents
原始文献 ↗ · BibTeX · RIS本次捕获版本报告多个接口存在脆弱性，所测防御保护有限；安全性需要与任务效用共同检查。
研究问题
智能体的哪些接口暴露出不同的攻击与防御失效模式？
设计与前提
ASB 区分系统与用户提示、工具响应和检索记忆，在共同智能体框架中评测攻击与防御。
研究发现
本次捕获版本报告多个接口存在脆弱性，所测防御保护有限；安全性需要与任务效用共同检查。
局限与反证
攻击面、基础模型和防御配置不同；汇总攻击率不是部署风险概率。方法与指标数量随版本变化，不能沿用旧摘要数字。
来源证据
| 系统 | 测量 | 原值 | 条件与位置 |
|---|---|---|---|
| Gemma2-9B | Attack success rate | 87.10%% | DPI#S5.T5; row Gemma2-9B; condition DPI; column Attack success rate |
| Gemma2-9B | Refusal rate | 4.30%% | DPI#S5.T5; row Gemma2-9B; condition DPI; column Refusal rate |
| Gemma2-9B | Attack success rate | 14.20%% | IPI#S5.T5; row Gemma2-9B; condition IPI; column Attack success rate |
| Gemma2-9B | Refusal rate | 15.00%% | IPI#S5.T5; row Gemma2-9B; condition IPI; column Refusal rate |
| Gemma2-9B | Attack success rate | 6.85%% | Memory poisoning#S5.T5; row Gemma2-9B; condition Memory poisoning; column Attack success rate |
| Gemma2-9B | Refusal rate | 9.85%% | Memory poisoning#S5.T5; row Gemma2-9B; condition Memory poisoning; column Refusal rate |
| Gemma2-9B | Attack success rate | 92.17%% | Mixed attack#S5.T5; row Gemma2-9B; condition Mixed attack; column Attack success rate |
| Gemma2-9B | Refusal rate | 1.33%% | Mixed attack#S5.T5; row Gemma2-9B; condition Mixed attack; column Refusal rate |
| Gemma2-9B | Attack success rate | 39.75%% | PoT backdoor#S5.T5; row Gemma2-9B; condition PoT backdoor; column Attack success rate |
| Gemma2-9B | Refusal rate | 5.25%% | PoT backdoor#S5.T5; row Gemma2-9B; condition PoT backdoor; column Refusal rate |
| Gemma2-9B | Attack success rate | 48.01%% | Average across attack types#S5.T5; row Gemma2-9B; condition Average across attack types; column Attack success rate |
| Gemma2-9B | Refusal rate | 7.15%% | Average across attack types#S5.T5; row Gemma2-9B; condition Average across attack types; column Refusal rate |
| Gemma2-27B | Attack success rate | 96.75%% | DPI#S5.T5; row Gemma2-27B; condition DPI; column Attack success rate |
| Gemma2-27B | Refusal rate | 0.90%% | DPI#S5.T5; row Gemma2-27B; condition DPI; column Refusal rate |
| Gemma2-27B | Attack success rate | 14.20%% | IPI#S5.T5; row Gemma2-27B; condition IPI; column Attack success rate |
| Gemma2-27B | Refusal rate | 3.90%% | IPI#S5.T5; row Gemma2-27B; condition IPI; column Refusal rate |
| Gemma2-27B | Attack success rate | 6.25%% | Memory poisoning#S5.T5; row Gemma2-27B; condition Memory poisoning; column Attack success rate |
| Gemma2-27B | Refusal rate | 5.45%% | Memory poisoning#S5.T5; row Gemma2-27B; condition Memory poisoning; column Refusal rate |
| Gemma2-27B | Attack success rate | 100.00%% | Mixed attack#S5.T5; row Gemma2-27B; condition Mixed attack; column Attack success rate |
| Gemma2-27B | Refusal rate | 0.50%% | Mixed attack#S5.T5; row Gemma2-27B; condition Mixed attack; column Refusal rate |
| Gemma2-27B | Attack success rate | 54.50%% | PoT backdoor#S5.T5; row Gemma2-27B; condition PoT backdoor; column Attack success rate |
| Gemma2-27B | Refusal rate | 3.50%% | PoT backdoor#S5.T5; row Gemma2-27B; condition PoT backdoor; column Refusal rate |
| Gemma2-27B | Attack success rate | 54.34%% | Average across attack types#S5.T5; row Gemma2-27B; condition Average across attack types; column Attack success rate |
| Gemma2-27B | Refusal rate | 2.85%% | Average across attack types#S5.T5; row Gemma2-27B; condition Average across attack types; column Refusal rate |
| LLaMA3-8B | Attack success rate | 25.20%% | DPI#S5.T5; row LLaMA3-8B; condition DPI; column Attack success rate |
| LLaMA3-8B | Refusal rate | 7.45%% | DPI#S5.T5; row LLaMA3-8B; condition DPI; column Refusal rate |
| LLaMA3-8B | Attack success rate | 10.55%% | IPI#S5.T5; row LLaMA3-8B; condition IPI; column Attack success rate |
| LLaMA3-8B | Refusal rate | 3.00%% | IPI#S5.T5; row LLaMA3-8B; condition IPI; column Refusal rate |
| LLaMA3-8B | Attack success rate | 3.30%% | Memory poisoning#S5.T5; row LLaMA3-8B; condition Memory poisoning; column Attack success rate |
| LLaMA3-8B | Refusal rate | 5.45%% | Memory poisoning#S5.T5; row LLaMA3-8B; condition Memory poisoning; column Refusal rate |
| LLaMA3-8B | Attack success rate | 40.75%% | Mixed attack#S5.T5; row LLaMA3-8B; condition Mixed attack; column Attack success rate |
| LLaMA3-8B | Refusal rate | 5.75%% | Mixed attack#S5.T5; row LLaMA3-8B; condition Mixed attack; column Refusal rate |
| LLaMA3-8B | Attack success rate | 21.50%% | PoT backdoor#S5.T5; row LLaMA3-8B; condition PoT backdoor; column Attack success rate |
| LLaMA3-8B | Refusal rate | 2.50%% | PoT backdoor#S5.T5; row LLaMA3-8B; condition PoT backdoor; column Refusal rate |
| LLaMA3-8B | Attack success rate | 20.26%% | Average across attack types#S5.T5; row LLaMA3-8B; condition Average across attack types; column Attack success rate |
| LLaMA3-8B | Refusal rate | 4.83%% | Average across attack types#S5.T5; row LLaMA3-8B; condition Average across attack types; column Refusal rate |
| LLaMA3-70B | Attack success rate | 86.15%% | DPI#S5.T5; row LLaMA3-70B; condition DPI; column Attack success rate |
| LLaMA3-70B | Refusal rate | 7.80%% | DPI#S5.T5; row LLaMA3-70B; condition DPI; column Refusal rate |
| LLaMA3-70B | Attack success rate | 43.70%% | IPI#S5.T5; row LLaMA3-70B; condition IPI; column Attack success rate |
| LLaMA3-70B | Refusal rate | 3.00%% | IPI#S5.T5; row LLaMA3-70B; condition IPI; column Refusal rate |
| LLaMA3-70B | Attack success rate | 1.85%% | Memory poisoning#S5.T5; row LLaMA3-70B; condition Memory poisoning; column Attack success rate |
| LLaMA3-70B | Refusal rate | 1.80%% | Memory poisoning#S5.T5; row LLaMA3-70B; condition Memory poisoning; column Refusal rate |
| LLaMA3-70B | Attack success rate | 85.50%% | Mixed attack#S5.T5; row LLaMA3-70B; condition Mixed attack; column Attack success rate |
| LLaMA3-70B | Refusal rate | 6.50%% | Mixed attack#S5.T5; row LLaMA3-70B; condition Mixed attack; column Refusal rate |
| LLaMA3-70B | Attack success rate | 57.00%% | PoT backdoor#S5.T5; row LLaMA3-70B; condition PoT backdoor; column Attack success rate |
| LLaMA3-70B | Refusal rate | 2.00%% | PoT backdoor#S5.T5; row LLaMA3-70B; condition PoT backdoor; column Refusal rate |
| LLaMA3-70B | Attack success rate | 54.84%% | Average across attack types#S5.T5; row LLaMA3-70B; condition Average across attack types; column Attack success rate |
| LLaMA3-70B | Refusal rate | 4.22%% | Average across attack types#S5.T5; row LLaMA3-70B; condition Average across attack types; column Refusal rate |
| LLaMA3.1-8B | Attack success rate | 51.10%% | DPI#S5.T5; row LLaMA3.1-8B; condition DPI; column Attack success rate |
| LLaMA3.1-8B | Refusal rate | 5.20%% | DPI#S5.T5; row LLaMA3.1-8B; condition DPI; column Refusal rate |
| LLaMA3.1-8B | Attack success rate | 6.40%% | IPI#S5.T5; row LLaMA3.1-8B; condition IPI; column Attack success rate |
| LLaMA3.1-8B | Refusal rate | 1.85%% | IPI#S5.T5; row LLaMA3.1-8B; condition IPI; column Refusal rate |
| LLaMA3.1-8B | Attack success rate | 25.65%% | Memory poisoning#S5.T5; row LLaMA3.1-8B; condition Memory poisoning; column Attack success rate |
| LLaMA3.1-8B | Refusal rate | 6.75%% | Memory poisoning#S5.T5; row LLaMA3.1-8B; condition Memory poisoning; column Refusal rate |
| LLaMA3.1-8B | Attack success rate | 73.50%% | Mixed attack#S5.T5; row LLaMA3.1-8B; condition Mixed attack; column Attack success rate |
| LLaMA3.1-8B | Refusal rate | 3.50%% | Mixed attack#S5.T5; row LLaMA3.1-8B; condition Mixed attack; column Refusal rate |
| LLaMA3.1-8B | Attack success rate | 19.00%% | PoT backdoor#S5.T5; row LLaMA3.1-8B; condition PoT backdoor; column Attack success rate |
| LLaMA3.1-8B | Refusal rate | 5.75%% | PoT backdoor#S5.T5; row LLaMA3.1-8B; condition PoT backdoor; column Refusal rate |
| LLaMA3.1-8B | Attack success rate | 35.13%% | Average across attack types#S5.T5; row LLaMA3.1-8B; condition Average across attack types; column Attack success rate |
| LLaMA3.1-8B | Refusal rate | 4.61%% | Average across attack types#S5.T5; row LLaMA3.1-8B; condition Average across attack types; column Refusal rate |
| LLaMA3.1-70B | Attack success rate | 85.65%% | DPI#S5.T5; row LLaMA3.1-70B; condition DPI; column Attack success rate |
| LLaMA3.1-70B | Refusal rate | 5.30%% | DPI#S5.T5; row LLaMA3.1-70B; condition DPI; column Refusal rate |
| LLaMA3.1-70B | Attack success rate | 12.10%% | IPI#S5.T5; row LLaMA3.1-70B; condition IPI; column Attack success rate |
| LLaMA3.1-70B | Refusal rate | 4.95%% | IPI#S5.T5; row LLaMA3.1-70B; condition IPI; column Refusal rate |
| LLaMA3.1-70B | Attack success rate | 2.85%% | Memory poisoning#S5.T5; row LLaMA3.1-70B; condition Memory poisoning; column Attack success rate |
| LLaMA3.1-70B | Refusal rate | 2.20%% | Memory poisoning#S5.T5; row LLaMA3.1-70B; condition Memory poisoning; column Refusal rate |
| LLaMA3.1-70B | Attack success rate | 94.50%% | Mixed attack#S5.T5; row LLaMA3.1-70B; condition Mixed attack; column Attack success rate |
| LLaMA3.1-70B | Refusal rate | 1.25%% | Mixed attack#S5.T5; row LLaMA3.1-70B; condition Mixed attack; column Refusal rate |
| LLaMA3.1-70B | Attack success rate | 59.75%% | PoT backdoor#S5.T5; row LLaMA3.1-70B; condition PoT backdoor; column Attack success rate |
| LLaMA3.1-70B | Refusal rate | 6.25%% | PoT backdoor#S5.T5; row LLaMA3.1-70B; condition PoT backdoor; column Refusal rate |
| LLaMA3.1-70B | Attack success rate | 50.97%% | Average across attack types#S5.T5; row LLaMA3.1-70B; condition Average across attack types; column Attack success rate |
| LLaMA3.1-70B | Refusal rate | 3.99%% | Average across attack types#S5.T5; row LLaMA3.1-70B; condition Average across attack types; column Refusal rate |
| Mixtral-8x7B | Attack success rate | 25.85%% | DPI#S5.T5; row Mixtral-8x7B; condition DPI; column Attack success rate |
| Mixtral-8x7B | Refusal rate | 9.55%% | DPI#S5.T5; row Mixtral-8x7B; condition DPI; column Refusal rate |
| Mixtral-8x7B | Attack success rate | 4.80%% | IPI#S5.T5; row Mixtral-8x7B; condition IPI; column Attack success rate |
| Mixtral-8x7B | Refusal rate | 8.55%% | IPI#S5.T5; row Mixtral-8x7B; condition IPI; column Refusal rate |
| Mixtral-8x7B | Attack success rate | 4.90%% | Memory poisoning#S5.T5; row Mixtral-8x7B; condition Memory poisoning; column Attack success rate |
| Mixtral-8x7B | Refusal rate | 1.35%% | Memory poisoning#S5.T5; row Mixtral-8x7B; condition Memory poisoning; column Refusal rate |
| Mixtral-8x7B | Attack success rate | 54.75%% | Mixed attack#S5.T5; row Mixtral-8x7B; condition Mixed attack; column Attack success rate |
| Mixtral-8x7B | Refusal rate | 6.75%% | Mixed attack#S5.T5; row Mixtral-8x7B; condition Mixed attack; column Refusal rate |
| Mixtral-8x7B | Attack success rate | 4.75%% | PoT backdoor#S5.T5; row Mixtral-8x7B; condition PoT backdoor; column Attack success rate |
| Mixtral-8x7B | Refusal rate | 13.25%% | PoT backdoor#S5.T5; row Mixtral-8x7B; condition PoT backdoor; column Refusal rate |
| Mixtral-8x7B | Attack success rate | 19.01%% | Average across attack types#S5.T5; row Mixtral-8x7B; condition Average across attack types; column Attack success rate |
| Mixtral-8x7B | Refusal rate | 7.89%% | Average across attack types#S5.T5; row Mixtral-8x7B; condition Average across attack types; column Refusal rate |
| Qwen2-7B | Attack success rate | 55.20%% | DPI#S5.T5; row Qwen2-7B; condition DPI; column Attack success rate |
| Qwen2-7B | Refusal rate | 7.70%% | DPI#S5.T5; row Qwen2-7B; condition DPI; column Refusal rate |
| Qwen2-7B | Attack success rate | 9.00%% | IPI#S5.T5; row Qwen2-7B; condition IPI; column Attack success rate |
| Qwen2-7B | Refusal rate | 6.00%% | IPI#S5.T5; row Qwen2-7B; condition IPI; column Refusal rate |
| Qwen2-7B | Attack success rate | 2.85%% | Memory poisoning#S5.T5; row Qwen2-7B; condition Memory poisoning; column Attack success rate |
| Qwen2-7B | Refusal rate | 4.95%% | Memory poisoning#S5.T5; row Qwen2-7B; condition Memory poisoning; column Refusal rate |
| Qwen2-7B | Attack success rate | 76.00%% | Mixed attack#S5.T5; row Qwen2-7B; condition Mixed attack; column Attack success rate |
| Qwen2-7B | Refusal rate | 2.50%% | Mixed attack#S5.T5; row Qwen2-7B; condition Mixed attack; column Refusal rate |
| Qwen2-7B | Attack success rate | 12.25%% | PoT backdoor#S5.T5; row Qwen2-7B; condition PoT backdoor; column Attack success rate |
| Qwen2-7B | Refusal rate | 4.50%% | PoT backdoor#S5.T5; row Qwen2-7B; condition PoT backdoor; column Refusal rate |
| Qwen2-7B | Attack success rate | 31.06%% | Average across attack types#S5.T5; row Qwen2-7B; condition Average across attack types; column Attack success rate |
| Qwen2-7B | Refusal rate | 5.13%% | Average across attack types#S5.T5; row Qwen2-7B; condition Average across attack types; column Refusal rate |
| Qwen2-72B | Attack success rate | 86.95%% | DPI#S5.T5; row Qwen2-72B; condition DPI; column Attack success rate |
| Qwen2-72B | Refusal rate | 4.20%% | DPI#S5.T5; row Qwen2-72B; condition DPI; column Refusal rate |
| Qwen2-72B | Attack success rate | 21.35%% | IPI#S5.T5; row Qwen2-72B; condition IPI; column Attack success rate |
| Qwen2-72B | Refusal rate | 16.55%% | IPI#S5.T5; row Qwen2-72B; condition IPI; column Refusal rate |
| Qwen2-72B | Attack success rate | 3.95%% | Memory poisoning#S5.T5; row Qwen2-72B; condition Memory poisoning; column Attack success rate |
| Qwen2-72B | Refusal rate | 5.45%% | Memory poisoning#S5.T5; row Qwen2-72B; condition Memory poisoning; column Refusal rate |
| Qwen2-72B | Attack success rate | 98.50%% | Mixed attack#S5.T5; row Qwen2-72B; condition Mixed attack; column Attack success rate |
| Qwen2-72B | Refusal rate | 0.75%% | Mixed attack#S5.T5; row Qwen2-72B; condition Mixed attack; column Refusal rate |
| Qwen2-72B | Attack success rate | 57.75%% | PoT backdoor#S5.T5; row Qwen2-72B; condition PoT backdoor; column Attack success rate |
| Qwen2-72B | Refusal rate | 4.75%% | PoT backdoor#S5.T5; row Qwen2-72B; condition PoT backdoor; column Refusal rate |
| Qwen2-72B | Attack success rate | 53.70%% | Average across attack types#S5.T5; row Qwen2-72B; condition Average across attack types; column Attack success rate |
| Qwen2-72B | Refusal rate | 6.34%% | Average across attack types#S5.T5; row Qwen2-72B; condition Average across attack types; column Refusal rate |
| Claude3.5 Sonnet | Attack success rate | 90.75%% | DPI#S5.T5; row Claude3.5 Sonnet; condition DPI; column Attack success rate |
| Claude3.5 Sonnet | Refusal rate | 7.65%% | DPI#S5.T5; row Claude3.5 Sonnet; condition DPI; column Refusal rate |
| Claude3.5 Sonnet | Attack success rate | 59.70%% | IPI#S5.T5; row Claude3.5 Sonnet; condition IPI; column Attack success rate |
| Claude3.5 Sonnet | Refusal rate | 25.50%% | IPI#S5.T5; row Claude3.5 Sonnet; condition IPI; column Refusal rate |
| Claude3.5 Sonnet | Attack success rate | 19.75%% | Memory poisoning#S5.T5; row Claude3.5 Sonnet; condition Memory poisoning; column Attack success rate |
| Claude3.5 Sonnet | Refusal rate | 1.20%% | Memory poisoning#S5.T5; row Claude3.5 Sonnet; condition Memory poisoning; column Refusal rate |
| Claude3.5 Sonnet | Attack success rate | 94.50%% | Mixed attack#S5.T5; row Claude3.5 Sonnet; condition Mixed attack; column Attack success rate |
| Claude3.5 Sonnet | Refusal rate | 6.25%% | Mixed attack#S5.T5; row Claude3.5 Sonnet; condition Mixed attack; column Refusal rate |
| Claude3.5 Sonnet | Attack success rate | 17.50%% | PoT backdoor#S5.T5; row Claude3.5 Sonnet; condition PoT backdoor; column Attack success rate |
| Claude3.5 Sonnet | Refusal rate | 11.75%% | PoT backdoor#S5.T5; row Claude3.5 Sonnet; condition PoT backdoor; column Refusal rate |
| Claude3.5 Sonnet | Attack success rate | 56.44%% | Average across attack types#S5.T5; row Claude3.5 Sonnet; condition Average across attack types; column Attack success rate |
| Claude3.5 Sonnet | Refusal rate | 10.47%% | Average across attack types#S5.T5; row Claude3.5 Sonnet; condition Average across attack types; column Refusal rate |
| GPT-3.5 Turbo | Attack success rate | 98.40%% | DPI#S5.T5; row GPT-3.5 Turbo; condition DPI; column Attack success rate |
| GPT-3.5 Turbo | Refusal rate | 3.00%% | DPI#S5.T5; row GPT-3.5 Turbo; condition DPI; column Refusal rate |
| GPT-3.5 Turbo | Attack success rate | 55.10%% | IPI#S5.T5; row GPT-3.5 Turbo; condition IPI; column Attack success rate |
| GPT-3.5 Turbo | Refusal rate | 16.85%% | IPI#S5.T5; row GPT-3.5 Turbo; condition IPI; column Refusal rate |
| GPT-3.5 Turbo | Attack success rate | 9.30%% | Memory poisoning#S5.T5; row GPT-3.5 Turbo; condition Memory poisoning; column Attack success rate |
| GPT-3.5 Turbo | Refusal rate | 0.30%% | Memory poisoning#S5.T5; row GPT-3.5 Turbo; condition Memory poisoning; column Refusal rate |
| GPT-3.5 Turbo | Attack success rate | 99.75%% | Mixed attack#S5.T5; row GPT-3.5 Turbo; condition Mixed attack; column Attack success rate |
| GPT-3.5 Turbo | Refusal rate | 0.00%% | Mixed attack#S5.T5; row GPT-3.5 Turbo; condition Mixed attack; column Refusal rate |
| GPT-3.5 Turbo | Attack success rate | 8.25%% | PoT backdoor#S5.T5; row GPT-3.5 Turbo; condition PoT backdoor; column Attack success rate |
| GPT-3.5 Turbo | Refusal rate | 10.75%% | PoT backdoor#S5.T5; row GPT-3.5 Turbo; condition PoT backdoor; column Refusal rate |
| GPT-3.5 Turbo | Attack success rate | 54.16%% | Average across attack types#S5.T5; row GPT-3.5 Turbo; condition Average across attack types; column Attack success rate |
| GPT-3.5 Turbo | Refusal rate | 6.18%% | Average across attack types#S5.T5; row GPT-3.5 Turbo; condition Average across attack types; column Refusal rate |
| GPT-4o | Attack success rate | 60.35%% | DPI#S5.T5; row GPT-4o; condition DPI; column Attack success rate |
| GPT-4o | Refusal rate | 20.05%% | DPI#S5.T5; row GPT-4o; condition DPI; column Refusal rate |
| GPT-4o | Attack success rate | 62.45%% | IPI#S5.T5; row GPT-4o; condition IPI; column Attack success rate |
| GPT-4o | Refusal rate | 6.50%% | IPI#S5.T5; row GPT-4o; condition IPI; column Refusal rate |
| GPT-4o | Attack success rate | 10.00%% | Memory poisoning#S5.T5; row GPT-4o; condition Memory poisoning; column Attack success rate |
| GPT-4o | Refusal rate | 11.75%% | Memory poisoning#S5.T5; row GPT-4o; condition Memory poisoning; column Refusal rate |
| GPT-4o | Attack success rate | 89.25%% | Mixed attack#S5.T5; row GPT-4o; condition Mixed attack; column Attack success rate |
| GPT-4o | Refusal rate | 5.50%% | Mixed attack#S5.T5; row GPT-4o; condition Mixed attack; column Refusal rate |
| GPT-4o | Attack success rate | 100.00%% | PoT backdoor#S5.T5; row GPT-4o; condition PoT backdoor; column Attack success rate |
| GPT-4o | Refusal rate | 0.25%% | PoT backdoor#S5.T5; row GPT-4o; condition PoT backdoor; column Refusal rate |
| GPT-4o | Attack success rate | 64.41%% | Average across attack types#S5.T5; row GPT-4o; condition Average across attack types; column Attack success rate |
| GPT-4o | Refusal rate | 8.81%% | Average across attack types#S5.T5; row GPT-4o; condition Average across attack types; column Refusal rate |
| GPT-4o-mini | Attack success rate | 95.45%% | DPI#S5.T5; row GPT-4o-mini; condition DPI; column Attack success rate |
| GPT-4o-mini | Refusal rate | 1.85%% | DPI#S5.T5; row GPT-4o-mini; condition DPI; column Refusal rate |
| GPT-4o-mini | Attack success rate | 44.55%% | IPI#S5.T5; row GPT-4o-mini; condition IPI; column Attack success rate |
| GPT-4o-mini | Refusal rate | 0.25%% | IPI#S5.T5; row GPT-4o-mini; condition IPI; column Refusal rate |
| GPT-4o-mini | Attack success rate | 5.50%% | Memory poisoning#S5.T5; row GPT-4o-mini; condition Memory poisoning; column Attack success rate |
| GPT-4o-mini | Refusal rate | 3.65%% | Memory poisoning#S5.T5; row GPT-4o-mini; condition Memory poisoning; column Refusal rate |
| GPT-4o-mini | Attack success rate | 96.75%% | Mixed attack#S5.T5; row GPT-4o-mini; condition Mixed attack; column Attack success rate |
| GPT-4o-mini | Refusal rate | 1.25%% | Mixed attack#S5.T5; row GPT-4o-mini; condition Mixed attack; column Refusal rate |
| GPT-4o-mini | Attack success rate | 95.50%% | PoT backdoor#S5.T5; row GPT-4o-mini; condition PoT backdoor; column Attack success rate |
| GPT-4o-mini | Refusal rate | 0.00%% | PoT backdoor#S5.T5; row GPT-4o-mini; condition PoT backdoor; column Refusal rate |
| GPT-4o-mini | Attack success rate | 67.55%% | Average across attack types#S5.T5; row GPT-4o-mini; condition Average across attack types; column Attack success rate |
| GPT-4o-mini | Refusal rate | 1.40%% | Average across attack types#S5.T5; row GPT-4o-mini; condition Average across attack types; column Refusal rate |
| Average | Attack success rate | 72.68%% | DPI#S5.T5; row Average; condition DPI; column Attack success rate |
| Average | Refusal rate | 6.53%% | DPI#S5.T5; row Average; condition DPI; column Refusal rate |
| Average | Attack success rate | 27.55%% | IPI#S5.T5; row Average; condition IPI; column Attack success rate |
| Average | Refusal rate | 8.61%% | IPI#S5.T5; row Average; condition IPI; column Refusal rate |
| Average | Attack success rate | 7.92%% | Memory poisoning#S5.T5; row Average; condition Memory poisoning; column Attack success rate |
| Average | Refusal rate | 4.63%% | Memory poisoning#S5.T5; row Average; condition Memory poisoning; column Refusal rate |
| Average | Attack success rate | 84.30%% | Mixed attack#S5.T5; row Average; condition Mixed attack; column Attack success rate |
| Average | Refusal rate | 3.22%% | Mixed attack#S5.T5; row Average; condition Mixed attack; column Refusal rate |
| Average | Attack success rate | 42.12%% | PoT backdoor#S5.T5; row Average; condition PoT backdoor; column Attack success rate |
| Average | Refusal rate | 5.42%% | PoT backdoor#S5.T5; row Average; condition PoT backdoor; column Refusal rate |
| Average | Attack success rate | 46.91%% | Average across attack types#S5.T5; row Average; condition Average across attack types; column Attack success rate |
| Average | Refusal rate | 5.68%% | Average across attack types#S5.T5; row Average; condition Average across attack types; column Refusal rate |
| Claude-3.5 Sonnet | Performance under no attack | 100.00% | No attack#S5.T6; row Claude-3.5 Sonnet; condition No attack; column Performance under no attack |
| Claude-3.5 Sonnet | Attack success rate | 56.44% | Average across attack types#S5.T6; row Claude-3.5 Sonnet; condition Average across attack types; column Attack success rate |
| Claude-3.5 Sonnet | NRP | 43.56% | Normalized risk-adjusted performance#S5.T6; row Claude-3.5 Sonnet; condition Normalized risk-adjusted performance; column NRP |
| LLaMA3-70B | Performance under no attack | 66.50% | No attack#S5.T6; row LLaMA3-70B; condition No attack; column Performance under no attack |
| LLaMA3-70B | Attack success rate | 54.84% | Average across attack types#S5.T6; row LLaMA3-70B; condition Average across attack types; column Attack success rate |
| LLaMA3-70B | NRP | 30.03% | Normalized risk-adjusted performance#S5.T6; row LLaMA3-70B; condition Normalized risk-adjusted performance; column NRP |
| GPT-4o | Performance under no attack | 79.00% | No attack#S5.T6; row GPT-4o; condition No attack; column Performance under no attack |
| GPT-4o | Attack success rate | 64.41% | Average across attack types#S5.T6; row GPT-4o; condition Average across attack types; column Attack success rate |
| GPT-4o | NRP | 28.12% | Normalized risk-adjusted performance#S5.T6; row GPT-4o; condition Normalized risk-adjusted performance; column NRP |
| GPT-4o-mini | Performance under no attack | 50.00% | No attack#S5.T6; row GPT-4o-mini; condition No attack; column Performance under no attack |
| GPT-4o-mini | Attack success rate | 67.55% | Average across attack types#S5.T6; row GPT-4o-mini; condition Average across attack types; column Attack success rate |
| GPT-4o-mini | NRP | 16.23% | Normalized risk-adjusted performance#S5.T6; row GPT-4o-mini; condition Normalized risk-adjusted performance; column NRP |
| Gemma2-27B | Performance under no attack | 31.50% | No attack#S5.T6; row Gemma2-27B; condition No attack; column Performance under no attack |
| Gemma2-27B | Attack success rate | 54.34% | Average across attack types#S5.T6; row Gemma2-27B; condition Average across attack types; column Attack success rate |
| Gemma2-27B | NRP | 14.38% | Normalized risk-adjusted performance#S5.T6; row Gemma2-27B; condition Normalized risk-adjusted performance; column NRP |
| LLaMA3.1-70B | Performance under no attack | 21.25% | No attack#S5.T6; row LLaMA3.1-70B; condition No attack; column Performance under no attack |
| LLaMA3.1-70B | Attack success rate | 50.97% | Average across attack types#S5.T6; row LLaMA3.1-70B; condition Average across attack types; column Attack success rate |
| LLaMA3.1-70B | NRP | 10.42% | Normalized risk-adjusted performance#S5.T6; row LLaMA3.1-70B; condition Normalized risk-adjusted performance; column NRP |
| Qwen2-7B | Performance under no attack | 9.75% | No attack#S5.T6; row Qwen2-7B; condition No attack; column Performance under no attack |
| Qwen2-7B | Attack success rate | 31.06% | Average across attack types#S5.T6; row Qwen2-7B; condition Average across attack types; column Attack success rate |
| Qwen2-7B | NRP | 6.72% | Normalized risk-adjusted performance#S5.T6; row Qwen2-7B; condition Normalized risk-adjusted performance; column NRP |
| Gemma2-9B | Performance under no attack | 10.75% | No attack#S5.T6; row Gemma2-9B; condition No attack; column Performance under no attack |
| Gemma2-9B | Attack success rate | 48.01% | Average across attack types#S5.T6; row Gemma2-9B; condition Average across attack types; column Attack success rate |
| Gemma2-9B | NRP | 5.59% | Normalized risk-adjusted performance#S5.T6; row Gemma2-9B; condition Normalized risk-adjusted performance; column NRP |
| GPT-3.5 Turbo | Performance under no attack | 8.00% | No attack#S5.T6; row GPT-3.5 Turbo; condition No attack; column Performance under no attack |
| GPT-3.5 Turbo | Attack success rate | 54.16% | Average across attack types#S5.T6; row GPT-3.5 Turbo; condition Average across attack types; column Attack success rate |
| GPT-3.5 Turbo | NRP | 3.67% | Normalized risk-adjusted performance#S5.T6; row GPT-3.5 Turbo; condition Normalized risk-adjusted performance; column NRP |
| Qwen2-72B | Performance under no attack | 4.00% | No attack#S5.T6; row Qwen2-72B; condition No attack; column Performance under no attack |
| Qwen2-72B | Attack success rate | 53.70% | Average across attack types#S5.T6; row Qwen2-72B; condition Average across attack types; column Attack success rate |
| Qwen2-72B | NRP | 1.85% | Normalized risk-adjusted performance#S5.T6; row Qwen2-72B; condition Normalized risk-adjusted performance; column NRP |
| LLaMA3-8B | Performance under no attack | 1.50% | No attack#S5.T6; row LLaMA3-8B; condition No attack; column Performance under no attack |
| LLaMA3-8B | Attack success rate | 20.26% | Average across attack types#S5.T6; row LLaMA3-8B; condition Average across attack types; column Attack success rate |
| LLaMA3-8B | NRP | 1.20% | Normalized risk-adjusted performance#S5.T6; row LLaMA3-8B; condition Normalized risk-adjusted performance; column NRP |
| LLaMA3.1-8B | Performance under no attack | 0.75% | No attack#S5.T6; row LLaMA3.1-8B; condition No attack; column Performance under no attack |
| LLaMA3.1-8B | Attack success rate | 35.13% | Average across attack types#S5.T6; row LLaMA3.1-8B; condition Average across attack types; column Attack success rate |
| LLaMA3.1-8B | NRP | 0.49% | Normalized risk-adjusted performance#S5.T6; row LLaMA3.1-8B; condition Normalized risk-adjusted performance; column NRP |
| Mixtral-8x7B | Performance under no attack | 0.00% | No attack#S5.T6; row Mixtral-8x7B; condition No attack; column Performance under no attack |
| Mixtral-8x7B | Attack success rate | 19.01% | Average across attack types#S5.T6; row Mixtral-8x7B; condition Average across attack types; column Attack success rate |
| Mixtral-8x7B | NRP | 0.00% | Normalized risk-adjusted performance#S5.T6; row Mixtral-8x7B; condition Normalized risk-adjusted performance; column NRP |
| Average | Performance under no attack | 29.46% | No attack#S5.T6; row Average; condition No attack; column Performance under no attack |
| Average | Attack success rate | 46.91% | Average across attack types#S5.T6; row Average; condition Average across attack types; column Attack success rate |
| Average | NRP | 15.64% | Normalized risk-adjusted performance#S5.T6; row Average; condition Normalized risk-adjusted performance; column NRP |
| Gemma2-9B | Attack success rate | 91.00%% | DPI baseline#S5.T8; row Gemma2-9B; condition DPI baseline; column Attack success rate |
| Gemma2-9B | Attack success rate | 91.75%% | Delimiter#S5.T8; row Gemma2-9B; condition Delimiter; column Attack success rate |
| Gemma2-9B | Attack success rate | 62.50%% | Paraphrase#S5.T8; row Gemma2-9B; condition Paraphrase; column Attack success rate |
| Gemma2-9B | Attack success rate | 91.00%% | Instructional#S5.T8; row Gemma2-9B; condition Instructional; column Attack success rate |
| Gemma2-9B | Attack success rate | 49.50%% | Dynamic prompt rewriting#S5.T8; row Gemma2-9B; condition Dynamic prompt rewriting; column Attack success rate |
| Gemma2-27B | Attack success rate | 98.75%% | DPI baseline#S5.T8; row Gemma2-27B; condition DPI baseline; column Attack success rate |
| Gemma2-27B | Attack success rate | 99.75%% | Delimiter#S5.T8; row Gemma2-27B; condition Delimiter; column Attack success rate |
| Gemma2-27B | Attack success rate | 68.00%% | Paraphrase#S5.T8; row Gemma2-27B; condition Paraphrase; column Attack success rate |
| Gemma2-27B | Attack success rate | 99.50%% | Instructional#S5.T8; row Gemma2-27B; condition Instructional; column Attack success rate |
| Gemma2-27B | Attack success rate | 67.50%% | Dynamic prompt rewriting#S5.T8; row Gemma2-27B; condition Dynamic prompt rewriting; column Attack success rate |
| LLaMA3-8B | Attack success rate | 33.75%% | DPI baseline#S5.T8; row LLaMA3-8B; condition DPI baseline; column Attack success rate |
| LLaMA3-8B | Attack success rate | 62.75%% | Delimiter#S5.T8; row LLaMA3-8B; condition Delimiter; column Attack success rate |
| LLaMA3-8B | Attack success rate | 28.50%% | Paraphrase#S5.T8; row LLaMA3-8B; condition Paraphrase; column Attack success rate |
| LLaMA3-8B | Attack success rate | 52.00%% | Instructional#S5.T8; row LLaMA3-8B; condition Instructional; column Attack success rate |
| LLaMA3-8B | Attack success rate | 23.15%% | Dynamic prompt rewriting#S5.T8; row LLaMA3-8B; condition Dynamic prompt rewriting; column Attack success rate |
| LLaMA3-70B | Attack success rate | 87.75%% | DPI baseline#S5.T8; row LLaMA3-70B; condition DPI baseline; column Attack success rate |
| LLaMA3-70B | Attack success rate | 88.25%% | Delimiter#S5.T8; row LLaMA3-70B; condition Delimiter; column Attack success rate |
| LLaMA3-70B | Attack success rate | 71.25%% | Paraphrase#S5.T8; row LLaMA3-70B; condition Paraphrase; column Attack success rate |
| LLaMA3-70B | Attack success rate | 87.25%% | Instructional#S5.T8; row LLaMA3-70B; condition Instructional; column Attack success rate |
| LLaMA3-70B | Attack success rate | 74.50%% | Dynamic prompt rewriting#S5.T8; row LLaMA3-70B; condition Dynamic prompt rewriting; column Attack success rate |
| LLaMA3.1-8B | Attack success rate | 64.25%% | DPI baseline#S5.T8; row LLaMA3.1-8B; condition DPI baseline; column Attack success rate |
| LLaMA3.1-8B | Attack success rate | 65.00%% | Delimiter#S5.T8; row LLaMA3.1-8B; condition Delimiter; column Attack success rate |
| LLaMA3.1-8B | Attack success rate | 42.50%% | Paraphrase#S5.T8; row LLaMA3.1-8B; condition Paraphrase; column Attack success rate |
| LLaMA3.1-8B | Attack success rate | 68.75%% | Instructional#S5.T8; row LLaMA3.1-8B; condition Instructional; column Attack success rate |
| LLaMA3.1-8B | Attack success rate | 26.70%% | Dynamic prompt rewriting#S5.T8; row LLaMA3.1-8B; condition Dynamic prompt rewriting; column Attack success rate |
| LLaMA3.1-70B | Attack success rate | 93.50%% | DPI baseline#S5.T8; row LLaMA3.1-70B; condition DPI baseline; column Attack success rate |
| LLaMA3.1-70B | Attack success rate | 92.75%% | Delimiter#S5.T8; row LLaMA3.1-70B; condition Delimiter; column Attack success rate |
| LLaMA3.1-70B | Attack success rate | 56.75%% | Paraphrase#S5.T8; row LLaMA3.1-70B; condition Paraphrase; column Attack success rate |
| LLaMA3.1-70B | Attack success rate | 90.50%% | Instructional#S5.T8; row LLaMA3.1-70B; condition Instructional; column Attack success rate |
| LLaMA3.1-70B | Attack success rate | 44.60%% | Dynamic prompt rewriting#S5.T8; row LLaMA3.1-70B; condition Dynamic prompt rewriting; column Attack success rate |
| Mixtral-8x7B | Attack success rate | 43.25%% | DPI baseline#S5.T8; row Mixtral-8x7B; condition DPI baseline; column Attack success rate |
| Mixtral-8x7B | Attack success rate | 43.00%% | Delimiter#S5.T8; row Mixtral-8x7B; condition Delimiter; column Attack success rate |
| Mixtral-8x7B | Attack success rate | 21.00%% | Paraphrase#S5.T8; row Mixtral-8x7B; condition Paraphrase; column Attack success rate |
| Mixtral-8x7B | Attack success rate | 34.00%% | Instructional#S5.T8; row Mixtral-8x7B; condition Instructional; column Attack success rate |
| Mixtral-8x7B | Attack success rate | 8.35%% | Dynamic prompt rewriting#S5.T8; row Mixtral-8x7B; condition Dynamic prompt rewriting; column Attack success rate |
| Qwen2-7B | Attack success rate | 73.50%% | DPI baseline#S5.T8; row Qwen2-7B; condition DPI baseline; column Attack success rate |
| Qwen2-7B | Attack success rate | 80.00%% | Delimiter#S5.T8; row Qwen2-7B; condition Delimiter; column Attack success rate |
| Qwen2-7B | Attack success rate | 46.25%% | Paraphrase#S5.T8; row Qwen2-7B; condition Paraphrase; column Attack success rate |
| Qwen2-7B | Attack success rate | 76.75%% | Instructional#S5.T8; row Qwen2-7B; condition Instructional; column Attack success rate |
| Qwen2-7B | Attack success rate | 33.30%% | Dynamic prompt rewriting#S5.T8; row Qwen2-7B; condition Dynamic prompt rewriting; column Attack success rate |
| Qwen2-72B | Attack success rate | 94.50%% | DPI baseline#S5.T8; row Qwen2-72B; condition DPI baseline; column Attack success rate |
| Qwen2-72B | Attack success rate | 95.00%% | Delimiter#S5.T8; row Qwen2-72B; condition Delimiter; column Attack success rate |
| Qwen2-72B | Attack success rate | 60.50%% | Paraphrase#S5.T8; row Qwen2-72B; condition Paraphrase; column Attack success rate |
| Qwen2-72B | Attack success rate | 95.50%% | Instructional#S5.T8; row Qwen2-72B; condition Instructional; column Attack success rate |
| Qwen2-72B | Attack success rate | 39.40%% | Dynamic prompt rewriting#S5.T8; row Qwen2-72B; condition Dynamic prompt rewriting; column Attack success rate |
| Claude-3.5 Sonnet | Attack success rate | 87.75%% | DPI baseline#S5.T8; row Claude-3.5 Sonnet; condition DPI baseline; column Attack success rate |
| Claude-3.5 Sonnet | Attack success rate | 79.00%% | Delimiter#S5.T8; row Claude-3.5 Sonnet; condition Delimiter; column Attack success rate |
| Claude-3.5 Sonnet | Attack success rate | 65.25%% | Paraphrase#S5.T8; row Claude-3.5 Sonnet; condition Paraphrase; column Attack success rate |
| Claude-3.5 Sonnet | Attack success rate | 70.25%% | Instructional#S5.T8; row Claude-3.5 Sonnet; condition Instructional; column Attack success rate |
| Claude-3.5 Sonnet | Attack success rate | 35.80%% | Dynamic prompt rewriting#S5.T8; row Claude-3.5 Sonnet; condition Dynamic prompt rewriting; column Attack success rate |
| GPT-3.5 Turbo | Attack success rate | 99.75%% | DPI baseline#S5.T8; row GPT-3.5 Turbo; condition DPI baseline; column Attack success rate |
| GPT-3.5 Turbo | Attack success rate | 99.75%% | Delimiter#S5.T8; row GPT-3.5 Turbo; condition Delimiter; column Attack success rate |
| GPT-3.5 Turbo | Attack success rate | 78.25%% | Paraphrase#S5.T8; row GPT-3.5 Turbo; condition Paraphrase; column Attack success rate |
| GPT-3.5 Turbo | Attack success rate | 99.50%% | Instructional#S5.T8; row GPT-3.5 Turbo; condition Instructional; column Attack success rate |
| GPT-3.5 Turbo | Attack success rate | 53.60%% | Dynamic prompt rewriting#S5.T8; row GPT-3.5 Turbo; condition Dynamic prompt rewriting; column Attack success rate |
| GPT-4o | Attack success rate | 55.50%% | DPI baseline#S5.T8; row GPT-4o; condition DPI baseline; column Attack success rate |
| GPT-4o | Attack success rate | 52.25%% | Delimiter#S5.T8; row GPT-4o; condition Delimiter; column Attack success rate |
| GPT-4o | Attack success rate | 62.50%% | Paraphrase#S5.T8; row GPT-4o; condition Paraphrase; column Attack success rate |
| GPT-4o | Attack success rate | 70.75%% | Instructional#S5.T8; row GPT-4o; condition Instructional; column Attack success rate |
| GPT-4o | Attack success rate | 60.20%% | Dynamic prompt rewriting#S5.T8; row GPT-4o; condition Dynamic prompt rewriting; column Attack success rate |
| GPT-4o-mini | Attack success rate | 95.75%% | DPI baseline#S5.T8; row GPT-4o-mini; condition DPI baseline; column Attack success rate |
| GPT-4o-mini | Attack success rate | 78.75%% | Delimiter#S5.T8; row GPT-4o-mini; condition Delimiter; column Attack success rate |
| GPT-4o-mini | Attack success rate | 76.00%% | Paraphrase#S5.T8; row GPT-4o-mini; condition Paraphrase; column Attack success rate |
| GPT-4o-mini | Attack success rate | 62.25%% | Instructional#S5.T8; row GPT-4o-mini; condition Instructional; column Attack success rate |
| GPT-4o-mini | Attack success rate | 61.25%% | Dynamic prompt rewriting#S5.T8; row GPT-4o-mini; condition Dynamic prompt rewriting; column Attack success rate |
| Average | Attack success rate | 78.38%% | DPI baseline#S5.T8; row Average; condition DPI baseline; column Attack success rate |
| Average | Attack success rate | 79.08%% | Delimiter#S5.T8; row Average; condition Delimiter; column Attack success rate |
| Average | Attack success rate | 56.87%% | Paraphrase#S5.T8; row Average; condition Paraphrase; column Attack success rate |
| Average | Attack success rate | 76.77%% | Instructional#S5.T8; row Average; condition Instructional; column Attack success rate |
| Average | Attack success rate | 44.45%% | Dynamic prompt rewriting#S5.T8; row Average; condition Dynamic prompt rewriting; column Attack success rate |
| Change versus table-specific baseline | Attack success rate | 0.00%percentage points | DPI baseline#S5.T8; row Δ; condition DPI baseline; column Attack success rate |
| Change versus table-specific baseline | Attack success rate | 0.69%percentage points | Delimiter#S5.T8; row Δ; condition Delimiter; column Attack success rate |
| Change versus table-specific baseline | Attack success rate | -21.52%percentage points | Paraphrase#S5.T8; row Δ; condition Paraphrase; column Attack success rate |
| Change versus table-specific baseline | Attack success rate | -1.62%percentage points | Instructional#S5.T8; row Δ; condition Instructional; column Attack success rate |
| Change versus table-specific baseline | Attack success rate | -33.93%percentage points | Dynamic prompt rewriting#S5.T8; row Δ; condition Dynamic prompt rewriting; column Attack success rate |
| Gemma2-9B | Attack success rate | 88.75%% | DPI+IPI#A4.T14; row Gemma2-9B; condition DPI+IPI; column Attack success rate |
| Gemma2-9B | Refusal rate | 3.75%% | DPI+IPI#A4.T14; row Gemma2-9B; condition DPI+IPI; column Refusal rate |
| Gemma2-9B | Attack success rate | 91.75%% | DPI+MP#A4.T14; row Gemma2-9B; condition DPI+MP; column Attack success rate |
| Gemma2-9B | Refusal rate | 1.75%% | DPI+MP#A4.T14; row Gemma2-9B; condition DPI+MP; column Refusal rate |
| Gemma2-9B | Attack success rate | 14.00%% | IPI+MP#A4.T14; row Gemma2-9B; condition IPI+MP; column Attack success rate |
| Gemma2-9B | Refusal rate | 26.75%% | IPI+MP#A4.T14; row Gemma2-9B; condition IPI+MP; column Refusal rate |
| Gemma2-9B | Attack success rate | 92.17%% | DPI+IPI+MP#A4.T14; row Gemma2-9B; condition DPI+IPI+MP; column Attack success rate |
| Gemma2-9B | Refusal rate | 1.33%% | DPI+IPI+MP#A4.T14; row Gemma2-9B; condition DPI+IPI+MP; column Refusal rate |
| Gemma2-27B | Attack success rate | 99.25%% | DPI+IPI#A4.T14; row Gemma2-27B; condition DPI+IPI; column Attack success rate |
| Gemma2-27B | Refusal rate | 0.00%% | DPI+IPI#A4.T14; row Gemma2-27B; condition DPI+IPI; column Refusal rate |
| Gemma2-27B | Attack success rate | 99.00%% | DPI+MP#A4.T14; row Gemma2-27B; condition DPI+MP; column Attack success rate |
| Gemma2-27B | Refusal rate | 1.00%% | DPI+MP#A4.T14; row Gemma2-27B; condition DPI+MP; column Refusal rate |
| Gemma2-27B | Attack success rate | 16.00%% | IPI+MP#A4.T14; row Gemma2-27B; condition IPI+MP; column Attack success rate |
| Gemma2-27B | Refusal rate | 3.25%% | IPI+MP#A4.T14; row Gemma2-27B; condition IPI+MP; column Refusal rate |
| Gemma2-27B | Attack success rate | 100.00%% | DPI+IPI+MP#A4.T14; row Gemma2-27B; condition DPI+IPI+MP; column Attack success rate |
| Gemma2-27B | Refusal rate | 0.50%% | DPI+IPI+MP#A4.T14; row Gemma2-27B; condition DPI+IPI+MP; column Refusal rate |
| LLaMA3-8B | Attack success rate | 55.75%% | DPI+IPI#A4.T14; row LLaMA3-8B; condition DPI+IPI; column Attack success rate |
| LLaMA3-8B | Refusal rate | 14.75%% | DPI+IPI#A4.T14; row LLaMA3-8B; condition DPI+IPI; column Refusal rate |
| LLaMA3-8B | Attack success rate | 37.75%% | DPI+MP#A4.T14; row LLaMA3-8B; condition DPI+MP; column Attack success rate |
| LLaMA3-8B | Refusal rate | 5.25%% | DPI+MP#A4.T14; row LLaMA3-8B; condition DPI+MP; column Refusal rate |
| LLaMA3-8B | Attack success rate | 4.25%% | IPI+MP#A4.T14; row LLaMA3-8B; condition IPI+MP; column Attack success rate |
| LLaMA3-8B | Refusal rate | 1.75%% | IPI+MP#A4.T14; row LLaMA3-8B; condition IPI+MP; column Refusal rate |
| LLaMA3-8B | Attack success rate | 40.75%% | DPI+IPI+MP#A4.T14; row LLaMA3-8B; condition DPI+IPI+MP; column Attack success rate |
| LLaMA3-8B | Refusal rate | 5.75%% | DPI+IPI+MP#A4.T14; row LLaMA3-8B; condition DPI+IPI+MP; column Refusal rate |
| LLaMA3-70B | Attack success rate | 86.50%% | DPI+IPI#A4.T14; row LLaMA3-70B; condition DPI+IPI; column Attack success rate |
| LLaMA3-70B | Refusal rate | 7.50%% | DPI+IPI#A4.T14; row LLaMA3-70B; condition DPI+IPI; column Refusal rate |
| LLaMA3-70B | Attack success rate | 83.75%% | DPI+MP#A4.T14; row LLaMA3-70B; condition DPI+MP; column Attack success rate |
| LLaMA3-70B | Refusal rate | 6.50%% | DPI+MP#A4.T14; row LLaMA3-70B; condition DPI+MP; column Refusal rate |
| LLaMA3-70B | Attack success rate | 27.50%% | IPI+MP#A4.T14; row LLaMA3-70B; condition IPI+MP; column Attack success rate |
| LLaMA3-70B | Refusal rate | 2.50%% | IPI+MP#A4.T14; row LLaMA3-70B; condition IPI+MP; column Refusal rate |
| LLaMA3-70B | Attack success rate | 85.50%% | DPI+IPI+MP#A4.T14; row LLaMA3-70B; condition DPI+IPI+MP; column Attack success rate |
| LLaMA3-70B | Refusal rate | 6.50%% | DPI+IPI+MP#A4.T14; row LLaMA3-70B; condition DPI+IPI+MP; column Refusal rate |
| LLaMA3.1-8B | Attack success rate | 68.25%% | DPI+IPI#A4.T14; row LLaMA3.1-8B; condition DPI+IPI; column Attack success rate |
| LLaMA3.1-8B | Refusal rate | 10.00%% | DPI+IPI#A4.T14; row LLaMA3.1-8B; condition DPI+IPI; column Refusal rate |
| LLaMA3.1-8B | Attack success rate | 67.50%% | DPI+MP#A4.T14; row LLaMA3.1-8B; condition DPI+MP; column Attack success rate |
| LLaMA3.1-8B | Refusal rate | 5.25%% | DPI+MP#A4.T14; row LLaMA3.1-8B; condition DPI+MP; column Refusal rate |
| LLaMA3.1-8B | Attack success rate | 6.25%% | IPI+MP#A4.T14; row LLaMA3.1-8B; condition IPI+MP; column Attack success rate |
| LLaMA3.1-8B | Refusal rate | 2.75%% | IPI+MP#A4.T14; row LLaMA3.1-8B; condition IPI+MP; column Refusal rate |
| LLaMA3.1-8B | Attack success rate | 73.50%% | DPI+IPI+MP#A4.T14; row LLaMA3.1-8B; condition DPI+IPI+MP; column Attack success rate |
| LLaMA3.1-8B | Refusal rate | 3.50%% | DPI+IPI+MP#A4.T14; row LLaMA3.1-8B; condition DPI+IPI+MP; column Refusal rate |
| LLaMA3.1-70B | Attack success rate | 88.00%% | DPI+IPI#A4.T14; row LLaMA3.1-70B; condition DPI+IPI; column Attack success rate |
| LLaMA3.1-70B | Refusal rate | 4.50%% | DPI+IPI#A4.T14; row LLaMA3.1-70B; condition DPI+IPI; column Refusal rate |
| LLaMA3.1-70B | Attack success rate | 95.75%% | DPI+MP#A4.T14; row LLaMA3.1-70B; condition DPI+MP; column Attack success rate |
| LLaMA3.1-70B | Refusal rate | 0.75%% | DPI+MP#A4.T14; row LLaMA3.1-70B; condition DPI+MP; column Refusal rate |
| LLaMA3.1-70B | Attack success rate | 10.25%% | IPI+MP#A4.T14; row LLaMA3.1-70B; condition IPI+MP; column Attack success rate |
| LLaMA3.1-70B | Refusal rate | 2.50%% | IPI+MP#A4.T14; row LLaMA3.1-70B; condition IPI+MP; column Refusal rate |
| LLaMA3.1-70B | Attack success rate | 94.50%% | DPI+IPI+MP#A4.T14; row LLaMA3.1-70B; condition DPI+IPI+MP; column Attack success rate |
| LLaMA3.1-70B | Refusal rate | 1.25%% | DPI+IPI+MP#A4.T14; row LLaMA3.1-70B; condition DPI+IPI+MP; column Refusal rate |
| Mixtral-8x7B | Attack success rate | 32.50%% | DPI+IPI#A4.T14; row Mixtral-8x7B; condition DPI+IPI; column Attack success rate |
| Mixtral-8x7B | Refusal rate | 11.25%% | DPI+IPI#A4.T14; row Mixtral-8x7B; condition DPI+IPI; column Refusal rate |
| Mixtral-8x7B | Attack success rate | 55.50%% | DPI+MP#A4.T14; row Mixtral-8x7B; condition DPI+MP; column Attack success rate |
| Mixtral-8x7B | Refusal rate | 7.75%% | DPI+MP#A4.T14; row Mixtral-8x7B; condition DPI+MP; column Refusal rate |
| Mixtral-8x7B | Attack success rate | 4.25%% | IPI+MP#A4.T14; row Mixtral-8x7B; condition IPI+MP; column Attack success rate |
| Mixtral-8x7B | Refusal rate | 5.50%% | IPI+MP#A4.T14; row Mixtral-8x7B; condition IPI+MP; column Refusal rate |
| Mixtral-8x7B | Attack success rate | 54.75%% | DPI+IPI+MP#A4.T14; row Mixtral-8x7B; condition DPI+IPI+MP; column Attack success rate |
| Mixtral-8x7B | Refusal rate | 6.75%% | DPI+IPI+MP#A4.T14; row Mixtral-8x7B; condition DPI+IPI+MP; column Refusal rate |
| Qwen2-7B | Attack success rate | 71.50%% | DPI+IPI#A4.T14; row Qwen2-7B; condition DPI+IPI; column Attack success rate |
| Qwen2-7B | Refusal rate | 6.50%% | DPI+IPI#A4.T14; row Qwen2-7B; condition DPI+IPI; column Refusal rate |
| Qwen2-7B | Attack success rate | 70.00%% | DPI+MP#A4.T14; row Qwen2-7B; condition DPI+MP; column Attack success rate |
| Qwen2-7B | Refusal rate | 1.50%% | DPI+MP#A4.T14; row Qwen2-7B; condition DPI+MP; column Refusal rate |
| Qwen2-7B | Attack success rate | 9.25%% | IPI+MP#A4.T14; row Qwen2-7B; condition IPI+MP; column Attack success rate |
| Qwen2-7B | Refusal rate | 7.00%% | IPI+MP#A4.T14; row Qwen2-7B; condition IPI+MP; column Refusal rate |
| Qwen2-7B | Attack success rate | 76.00%% | DPI+IPI+MP#A4.T14; row Qwen2-7B; condition DPI+IPI+MP; column Attack success rate |
| Qwen2-7B | Refusal rate | 2.50%% | DPI+IPI+MP#A4.T14; row Qwen2-7B; condition DPI+IPI+MP; column Refusal rate |
| Qwen2-72B | Attack success rate | 97.75%% | DPI+IPI#A4.T14; row Qwen2-72B; condition DPI+IPI; column Attack success rate |
| Qwen2-72B | Refusal rate | 1.75%% | DPI+IPI#A4.T14; row Qwen2-72B; condition DPI+IPI; column Refusal rate |
| Qwen2-72B | Attack success rate | 98.00%% | DPI+MP#A4.T14; row Qwen2-72B; condition DPI+MP; column Attack success rate |
| Qwen2-72B | Refusal rate | 3.00%% | DPI+MP#A4.T14; row Qwen2-72B; condition DPI+MP; column Refusal rate |
| Qwen2-72B | Attack success rate | 23.25%% | IPI+MP#A4.T14; row Qwen2-72B; condition IPI+MP; column Attack success rate |
| Qwen2-72B | Refusal rate | 1.75%% | IPI+MP#A4.T14; row Qwen2-72B; condition IPI+MP; column Refusal rate |
| Qwen2-72B | Attack success rate | 98.50%% | DPI+IPI+MP#A4.T14; row Qwen2-72B; condition DPI+IPI+MP; column Attack success rate |
| Qwen2-72B | Refusal rate | 0.75%% | DPI+IPI+MP#A4.T14; row Qwen2-72B; condition DPI+IPI+MP; column Refusal rate |
| Claude-3.5 Sonnet | Attack success rate | 88.00%% | DPI+IPI#A4.T14; row Claude-3.5 Sonnet; condition DPI+IPI; column Attack success rate |
| Claude-3.5 Sonnet | Refusal rate | 21.00%% | DPI+IPI#A4.T14; row Claude-3.5 Sonnet; condition DPI+IPI; column Refusal rate |
| Claude-3.5 Sonnet | Attack success rate | 94.50%% | DPI+MP#A4.T14; row Claude-3.5 Sonnet; condition DPI+MP; column Attack success rate |
| Claude-3.5 Sonnet | Refusal rate | 6.75%% | DPI+MP#A4.T14; row Claude-3.5 Sonnet; condition DPI+MP; column Refusal rate |
| Claude-3.5 Sonnet | Attack success rate | 58.25%% | IPI+MP#A4.T14; row Claude-3.5 Sonnet; condition IPI+MP; column Attack success rate |
| Claude-3.5 Sonnet | Refusal rate | 16.75%% | IPI+MP#A4.T14; row Claude-3.5 Sonnet; condition IPI+MP; column Refusal rate |
| Claude-3.5 Sonnet | Attack success rate | 94.50%% | DPI+IPI+MP#A4.T14; row Claude-3.5 Sonnet; condition DPI+IPI+MP; column Attack success rate |
| Claude-3.5 Sonnet | Refusal rate | 6.25%% | DPI+IPI+MP#A4.T14; row Claude-3.5 Sonnet; condition DPI+IPI+MP; column Refusal rate |
| GPT-3.5 Turbo | Attack success rate | 99.50%% | DPI+IPI#A4.T14; row GPT-3.5 Turbo; condition DPI+IPI; column Attack success rate |
| GPT-3.5 Turbo | Refusal rate | 0.00%% | DPI+IPI#A4.T14; row GPT-3.5 Turbo; condition DPI+IPI; column Refusal rate |
| GPT-3.5 Turbo | Attack success rate | 100.00%% | DPI+MP#A4.T14; row GPT-3.5 Turbo; condition DPI+MP; column Attack success rate |
| GPT-3.5 Turbo | Refusal rate | 0.25%% | DPI+MP#A4.T14; row GPT-3.5 Turbo; condition DPI+MP; column Refusal rate |
| GPT-3.5 Turbo | Attack success rate | 41.25%% | IPI+MP#A4.T14; row GPT-3.5 Turbo; condition IPI+MP; column Attack success rate |
| GPT-3.5 Turbo | Refusal rate | 12.75%% | IPI+MP#A4.T14; row GPT-3.5 Turbo; condition IPI+MP; column Refusal rate |
| GPT-3.5 Turbo | Attack success rate | 99.75%% | DPI+IPI+MP#A4.T14; row GPT-3.5 Turbo; condition DPI+IPI+MP; column Attack success rate |
| GPT-3.5 Turbo | Refusal rate | 0.00%% | DPI+IPI+MP#A4.T14; row GPT-3.5 Turbo; condition DPI+IPI+MP; column Refusal rate |
| GPT-4o | Attack success rate | 63.50%% | DPI+IPI#A4.T14; row GPT-4o; condition DPI+IPI; column Attack success rate |
| GPT-4o | Refusal rate | 19.25%% | DPI+IPI#A4.T14; row GPT-4o; condition DPI+IPI; column Refusal rate |
| GPT-4o | Attack success rate | 88.25%% | DPI+MP#A4.T14; row GPT-4o; condition DPI+MP; column Attack success rate |
| GPT-4o | Refusal rate | 7.50%% | DPI+MP#A4.T14; row GPT-4o; condition DPI+MP; column Refusal rate |
| GPT-4o | Attack success rate | 55.75%% | IPI+MP#A4.T14; row GPT-4o; condition IPI+MP; column Attack success rate |
| GPT-4o | Refusal rate | 0.75%% | IPI+MP#A4.T14; row GPT-4o; condition IPI+MP; column Refusal rate |
| GPT-4o | Attack success rate | 89.25%% | DPI+IPI+MP#A4.T14; row GPT-4o; condition DPI+IPI+MP; column Attack success rate |
| GPT-4o | Refusal rate | 5.50%% | DPI+IPI+MP#A4.T14; row GPT-4o; condition DPI+IPI+MP; column Refusal rate |
| GPT-4o-mini | Attack success rate | 88.50%% | DPI+IPI#A4.T14; row GPT-4o-mini; condition DPI+IPI; column Attack success rate |
| GPT-4o-mini | Refusal rate | 4.25%% | DPI+IPI#A4.T14; row GPT-4o-mini; condition DPI+IPI; column Refusal rate |
| GPT-4o-mini | Attack success rate | 97.50%% | DPI+MP#A4.T14; row GPT-4o-mini; condition DPI+MP; column Attack success rate |
| GPT-4o-mini | Refusal rate | 1.25%% | DPI+MP#A4.T14; row GPT-4o-mini; condition DPI+MP; column Refusal rate |
| GPT-4o-mini | Attack success rate | 38.50%% | IPI+MP#A4.T14; row GPT-4o-mini; condition IPI+MP; column Attack success rate |
| GPT-4o-mini | Refusal rate | 0.00%% | IPI+MP#A4.T14; row GPT-4o-mini; condition IPI+MP; column Refusal rate |
| GPT-4o-mini | Attack success rate | 96.75%% | DPI+IPI+MP#A4.T14; row GPT-4o-mini; condition DPI+IPI+MP; column Attack success rate |
| GPT-4o-mini | Refusal rate | 1.25%% | DPI+IPI+MP#A4.T14; row GPT-4o-mini; condition DPI+IPI+MP; column Refusal rate |
| Average | Attack success rate | 79.06%% | DPI+IPI#A4.T14; row Average; condition DPI+IPI; column Attack success rate |
| Average | Refusal rate | 8.04%% | DPI+IPI#A4.T14; row Average; condition DPI+IPI; column Refusal rate |
| Average | Attack success rate | 83.02%% | DPI+MP#A4.T14; row Average; condition DPI+MP; column Attack success rate |
| Average | Refusal rate | 3.73%% | DPI+MP#A4.T14; row Average; condition DPI+MP; column Refusal rate |
| Average | Attack success rate | 23.75%% | IPI+MP#A4.T14; row Average; condition IPI+MP; column Attack success rate |
| Average | Refusal rate | 6.46%% | IPI+MP#A4.T14; row Average; condition IPI+MP; column Refusal rate |
| Average | Attack success rate | 84.30%% | DPI+IPI+MP#A4.T14; row Average; condition DPI+IPI+MP; column Attack success rate |
| Average | Refusal rate | 3.22%% | DPI+IPI+MP#A4.T14; row Average; condition DPI+IPI+MP; column Refusal rate |
| Change versus table-specific baseline | Attack success rate | 6.37%percentage points | DPI+IPI#A4.T14; row Δ; condition DPI+IPI; column Attack success rate |
| Change versus table-specific baseline | Refusal rate | 1.50%percentage points | DPI+IPI#A4.T14; row Δ; condition DPI+IPI; column Refusal rate |
| Change versus table-specific baseline | Attack success rate | 10.33%percentage points | DPI+MP#A4.T14; row Δ; condition DPI+MP; column Attack success rate |
| Change versus table-specific baseline | Refusal rate | -2.80%percentage points | DPI+MP#A4.T14; row Δ; condition DPI+MP; column Refusal rate |
| Change versus table-specific baseline | Attack success rate | -3.80%percentage points | IPI+MP#A4.T14; row Δ; condition IPI+MP; column Attack success rate |
| Change versus table-specific baseline | Refusal rate | -2.15%percentage points | IPI+MP#A4.T14; row Δ; condition IPI+MP; column Refusal rate |
| Change versus table-specific baseline | Attack success rate | 11.62%percentage points | DPI+IPI+MP#A4.T14; row Δ; condition DPI+IPI+MP; column Attack success rate |
| Change versus table-specific baseline | Refusal rate | -3.32%percentage points | DPI+IPI+MP#A4.T14; row Δ; condition DPI+IPI+MP; column Refusal rate |
| Gemma2-9B | Benign performance | 21.00%% | PoT backdoor#A4.T16; row Gemma2-9B; condition PoT backdoor; column Benign performance |
| Gemma2-9B | Performance under no attack | 10.75%% | No attack#A4.T16; row Gemma2-9B; condition No attack; column Performance under no attack |
| Gemma2-27B | Benign performance | 37.75%% | PoT backdoor#A4.T16; row Gemma2-27B; condition PoT backdoor; column Benign performance |
| Gemma2-27B | Performance under no attack | 31.50%% | No attack#A4.T16; row Gemma2-27B; condition No attack; column Performance under no attack |
| LLaMA3-8B | Benign performance | 4.25%% | PoT backdoor#A4.T16; row LLaMA3-8B; condition PoT backdoor; column Benign performance |
| LLaMA3-8B | Performance under no attack | 1.50%% | No attack#A4.T16; row LLaMA3-8B; condition No attack; column Performance under no attack |
| LLaMA3-70B | Benign performance | 59.00%% | PoT backdoor#A4.T16; row LLaMA3-70B; condition PoT backdoor; column Benign performance |
| LLaMA3-70B | Performance under no attack | 66.50%% | No attack#A4.T16; row LLaMA3-70B; condition No attack; column Performance under no attack |
| LLaMA3.1-8B | Benign performance | 2.75%% | PoT backdoor#A4.T16; row LLaMA3.1-8B; condition PoT backdoor; column Benign performance |
| LLaMA3.1-8B | Performance under no attack | 0.75%% | No attack#A4.T16; row LLaMA3.1-8B; condition No attack; column Performance under no attack |
| LLaMA3.1-70B | Benign performance | 32.00%% | PoT backdoor#A4.T16; row LLaMA3.1-70B; condition PoT backdoor; column Benign performance |
| LLaMA3.1-70B | Performance under no attack | 21.25%% | No attack#A4.T16; row LLaMA3.1-70B; condition No attack; column Performance under no attack |
| Mixtral-8x7B | Benign performance | 1.00%% | PoT backdoor#A4.T16; row Mixtral-8x7B; condition PoT backdoor; column Benign performance |
| Mixtral-8x7B | Performance under no attack | 0.00%% | No attack#A4.T16; row Mixtral-8x7B; condition No attack; column Performance under no attack |
| Qwen2-7B | Benign performance | 8.50%% | PoT backdoor#A4.T16; row Qwen2-7B; condition PoT backdoor; column Benign performance |
| Qwen2-7B | Performance under no attack | 9.75%% | No attack#A4.T16; row Qwen2-7B; condition No attack; column Performance under no attack |
| Qwen2-72B | Benign performance | 2.25%% | PoT backdoor#A4.T16; row Qwen2-72B; condition PoT backdoor; column Benign performance |
| Qwen2-72B | Performance under no attack | 4.00%% | No attack#A4.T16; row Qwen2-72B; condition No attack; column Performance under no attack |
| Claude-3.5 Sonnet | Benign performance | 90.75%% | PoT backdoor#A4.T16; row Claude-3.5 Sonnet; condition PoT backdoor; column Benign performance |
| Claude-3.5 Sonnet | Performance under no attack | 100.00%% | No attack#A4.T16; row Claude-3.5 Sonnet; condition No attack; column Performance under no attack |
| GPT-3.5 Turbo | Benign performance | 17.25%% | PoT backdoor#A4.T16; row GPT-3.5 Turbo; condition PoT backdoor; column Benign performance |
| GPT-3.5 Turbo | Performance under no attack | 8.00%% | No attack#A4.T16; row GPT-3.5 Turbo; condition No attack; column Performance under no attack |
| GPT-4o | Benign performance | 77.75%% | PoT backdoor#A4.T16; row GPT-4o; condition PoT backdoor; column Benign performance |
| GPT-4o | Performance under no attack | 79.00%% | No attack#A4.T16; row GPT-4o; condition No attack; column Performance under no attack |
| GPT-4o-mini | Benign performance | 64.50%% | PoT backdoor#A4.T16; row GPT-4o-mini; condition PoT backdoor; column Benign performance |
| GPT-4o-mini | Performance under no attack | 50.00%% | No attack#A4.T16; row GPT-4o-mini; condition No attack; column Performance under no attack |
| Average | Benign performance | 32.21%% | PoT backdoor#A4.T16; row Average; condition PoT backdoor; column Benign performance |
| Average | Performance under no attack | 29.46%% | No attack#A4.T16; row Average; condition No attack; column Performance under no attack |
| Gemma2-9B | Performance under no attack | 10.75%% | No attack#A4.T19; row Gemma2-9B; condition No attack; column Performance under no attack |
| Gemma2-9B | Performance under no attack | 8.00%% | Delimiters#A4.T19; row Gemma2-9B; condition Delimiters; column Performance under no attack |
| Gemma2-9B | Change from no-defense PNA | -2.75%percentage points | Delimiters#A4.T19; row Gemma2-9B; condition Delimiters; column Change from no-defense PNA |
| Gemma2-9B | Performance under no attack | 9.00%% | Paraphrasing#A4.T19; row Gemma2-9B; condition Paraphrasing; column Performance under no attack |
| Gemma2-9B | Change from no-defense PNA | -1.75%percentage points | Paraphrasing#A4.T19; row Gemma2-9B; condition Paraphrasing; column Change from no-defense PNA |
| Gemma2-9B | Performance under no attack | 8.00%% | Instructional#A4.T19; row Gemma2-9B; condition Instructional; column Performance under no attack |
| Gemma2-9B | Change from no-defense PNA | -2.75%percentage points | Instructional#A4.T19; row Gemma2-9B; condition Instructional; column Change from no-defense PNA |
| Gemma2-9B | Performance under no attack | 14.00%% | Sandwich#A4.T19; row Gemma2-9B; condition Sandwich; column Performance under no attack |
| Gemma2-9B | Change from no-defense PNA | 3.25%percentage points | Sandwich#A4.T19; row Gemma2-9B; condition Sandwich; column Change from no-defense PNA |
| Gemma2-9B | Performance under no attack | 10.20%% | DPR#A4.T19; row Gemma2-9B; condition DPR; column Performance under no attack |
| Gemma2-9B | Change from no-defense PNA | -0.55%percentage points | DPR#A4.T19; row Gemma2-9B; condition DPR; column Change from no-defense PNA |
| Gemma2-27B | Performance under no attack | 31.50%% | No attack#A4.T19; row Gemma2-27B; condition No attack; column Performance under no attack |
| Gemma2-27B | Performance under no attack | 14.75%% | Delimiters#A4.T19; row Gemma2-27B; condition Delimiters; column Performance under no attack |
| Gemma2-27B | Change from no-defense PNA | -16.75%percentage points | Delimiters#A4.T19; row Gemma2-27B; condition Delimiters; column Change from no-defense PNA |
| Gemma2-27B | Performance under no attack | 24.75%% | Paraphrasing#A4.T19; row Gemma2-27B; condition Paraphrasing; column Performance under no attack |
| Gemma2-27B | Change from no-defense PNA | -6.75%percentage points | Paraphrasing#A4.T19; row Gemma2-27B; condition Paraphrasing; column Change from no-defense PNA |
| Gemma2-27B | Performance under no attack | 23.00%% | Instructional#A4.T19; row Gemma2-27B; condition Instructional; column Performance under no attack |
| Gemma2-27B | Change from no-defense PNA | -8.50%percentage points | Instructional#A4.T19; row Gemma2-27B; condition Instructional; column Change from no-defense PNA |
| Gemma2-27B | Performance under no attack | 27.00%% | Sandwich#A4.T19; row Gemma2-27B; condition Sandwich; column Performance under no attack |
| Gemma2-27B | Change from no-defense PNA | -4.50%percentage points | Sandwich#A4.T19; row Gemma2-27B; condition Sandwich; column Change from no-defense PNA |
| Gemma2-27B | Performance under no attack | 30.50%% | DPR#A4.T19; row Gemma2-27B; condition DPR; column Performance under no attack |
| Gemma2-27B | Change from no-defense PNA | -1.00%percentage points | DPR#A4.T19; row Gemma2-27B; condition DPR; column Change from no-defense PNA |
| LLaMA3-8B | Performance under no attack | 1.50%% | No attack#A4.T19; row LLaMA3-8B; condition No attack; column Performance under no attack |
| LLaMA3-8B | Performance under no attack | 1.75%% | Delimiters#A4.T19; row LLaMA3-8B; condition Delimiters; column Performance under no attack |
| LLaMA3-8B | Change from no-defense PNA | 0.25%percentage points | Delimiters#A4.T19; row LLaMA3-8B; condition Delimiters; column Change from no-defense PNA |
| LLaMA3-8B | Performance under no attack | 8.25%% | Paraphrasing#A4.T19; row LLaMA3-8B; condition Paraphrasing; column Performance under no attack |
| LLaMA3-8B | Change from no-defense PNA | 6.75%percentage points | Paraphrasing#A4.T19; row LLaMA3-8B; condition Paraphrasing; column Change from no-defense PNA |
| LLaMA3-8B | Performance under no attack | 5.00%% | Instructional#A4.T19; row LLaMA3-8B; condition Instructional; column Performance under no attack |
| LLaMA3-8B | Change from no-defense PNA | 3.50%percentage points | Instructional#A4.T19; row LLaMA3-8B; condition Instructional; column Change from no-defense PNA |
| LLaMA3-8B | Performance under no attack | 8.50%% | Sandwich#A4.T19; row LLaMA3-8B; condition Sandwich; column Performance under no attack |
| LLaMA3-8B | Change from no-defense PNA | 7.00%percentage points | Sandwich#A4.T19; row LLaMA3-8B; condition Sandwich; column Change from no-defense PNA |
| LLaMA3-8B | Performance under no attack | 7.50%% | DPR#A4.T19; row LLaMA3-8B; condition DPR; column Performance under no attack |
| LLaMA3-8B | Change from no-defense PNA | 6.00%percentage points | DPR#A4.T19; row LLaMA3-8B; condition DPR; column Change from no-defense PNA |
| LLaMA3-70B | Performance under no attack | 66.50%% | No attack#A4.T19; row LLaMA3-70B; condition No attack; column Performance under no attack |
| LLaMA3-70B | Performance under no attack | 58.50%% | Delimiters#A4.T19; row LLaMA3-70B; condition Delimiters; column Performance under no attack |
| LLaMA3-70B | Change from no-defense PNA | -8.00%percentage points | Delimiters#A4.T19; row LLaMA3-70B; condition Delimiters; column Change from no-defense PNA |
| LLaMA3-70B | Performance under no attack | 68.25%% | Paraphrasing#A4.T19; row LLaMA3-70B; condition Paraphrasing; column Performance under no attack |
| LLaMA3-70B | Change from no-defense PNA | 1.75%percentage points | Paraphrasing#A4.T19; row LLaMA3-70B; condition Paraphrasing; column Change from no-defense PNA |
| LLaMA3-70B | Performance under no attack | 70.00%% | Instructional#A4.T19; row LLaMA3-70B; condition Instructional; column Performance under no attack |
| LLaMA3-70B | Change from no-defense PNA | 3.50%percentage points | Instructional#A4.T19; row LLaMA3-70B; condition Instructional; column Change from no-defense PNA |
| LLaMA3-70B | Performance under no attack | 68.00%% | Sandwich#A4.T19; row LLaMA3-70B; condition Sandwich; column Performance under no attack |
| LLaMA3-70B | Change from no-defense PNA | 1.50%percentage points | Sandwich#A4.T19; row LLaMA3-70B; condition Sandwich; column Change from no-defense PNA |
| LLaMA3-70B | Performance under no attack | 64.50%% | DPR#A4.T19; row LLaMA3-70B; condition DPR; column Performance under no attack |
| LLaMA3-70B | Change from no-defense PNA | -2.00%percentage points | DPR#A4.T19; row LLaMA3-70B; condition DPR; column Change from no-defense PNA |
| LLaMA3.1-8B | Performance under no attack | 0.75%% | No attack#A4.T19; row LLaMA3.1-8B; condition No attack; column Performance under no attack |
| LLaMA3.1-8B | Performance under no attack | 0.00%% | Delimiters#A4.T19; row LLaMA3.1-8B; condition Delimiters; column Performance under no attack |
| LLaMA3.1-8B | Change from no-defense PNA | -0.75%percentage points | Delimiters#A4.T19; row LLaMA3.1-8B; condition Delimiters; column Change from no-defense PNA |
| LLaMA3.1-8B | Performance under no attack | 1.00%% | Paraphrasing#A4.T19; row LLaMA3.1-8B; condition Paraphrasing; column Performance under no attack |
| LLaMA3.1-8B | Change from no-defense PNA | 0.25%percentage points | Paraphrasing#A4.T19; row LLaMA3.1-8B; condition Paraphrasing; column Change from no-defense PNA |
| LLaMA3.1-8B | Performance under no attack | 0.75%% | Instructional#A4.T19; row LLaMA3.1-8B; condition Instructional; column Performance under no attack |
| LLaMA3.1-8B | Change from no-defense PNA | 0.00%percentage points | Instructional#A4.T19; row LLaMA3.1-8B; condition Instructional; column Change from no-defense PNA |
| LLaMA3.1-8B | Performance under no attack | 0.25%% | Sandwich#A4.T19; row LLaMA3.1-8B; condition Sandwich; column Performance under no attack |
| LLaMA3.1-8B | Change from no-defense PNA | -0.50%percentage points | Sandwich#A4.T19; row LLaMA3.1-8B; condition Sandwich; column Change from no-defense PNA |
| LLaMA3.1-8B | Performance under no attack | 0.50%% | DPR#A4.T19; row LLaMA3.1-8B; condition DPR; column Performance under no attack |
| LLaMA3.1-8B | Change from no-defense PNA | -0.25%percentage points | DPR#A4.T19; row LLaMA3.1-8B; condition DPR; column Change from no-defense PNA |
| LLaMA3.1-70B | Performance under no attack | 21.25%% | No attack#A4.T19; row LLaMA3.1-70B; condition No attack; column Performance under no attack |
| LLaMA3.1-70B | Performance under no attack | 9.50%% | Delimiters#A4.T19; row LLaMA3.1-70B; condition Delimiters; column Performance under no attack |
| LLaMA3.1-70B | Change from no-defense PNA | -11.75%percentage points | Delimiters#A4.T19; row LLaMA3.1-70B; condition Delimiters; column Change from no-defense PNA |
| LLaMA3.1-70B | Performance under no attack | 18.75%% | Paraphrasing#A4.T19; row LLaMA3.1-70B; condition Paraphrasing; column Performance under no attack |
| LLaMA3.1-70B | Change from no-defense PNA | -2.50%percentage points | Paraphrasing#A4.T19; row LLaMA3.1-70B; condition Paraphrasing; column Change from no-defense PNA |
| LLaMA3.1-70B | Performance under no attack | 14.50%% | Instructional#A4.T19; row LLaMA3.1-70B; condition Instructional; column Performance under no attack |
| LLaMA3.1-70B | Change from no-defense PNA | -6.75%percentage points | Instructional#A4.T19; row LLaMA3.1-70B; condition Instructional; column Change from no-defense PNA |
| LLaMA3.1-70B | Performance under no attack | 16.50%% | Sandwich#A4.T19; row LLaMA3.1-70B; condition Sandwich; column Performance under no attack |
| LLaMA3.1-70B | Change from no-defense PNA | -4.75%percentage points | Sandwich#A4.T19; row LLaMA3.1-70B; condition Sandwich; column Change from no-defense PNA |
| LLaMA3.1-70B | Performance under no attack | 13.80%% | DPR#A4.T19; row LLaMA3.1-70B; condition DPR; column Performance under no attack |
| LLaMA3.1-70B | Change from no-defense PNA | -7.45%percentage points | DPR#A4.T19; row LLaMA3.1-70B; condition DPR; column Change from no-defense PNA |
| Mixtral-8x7B | Performance under no attack | 0.00%% | No attack#A4.T19; row Mixtral-8x7B; condition No attack; column Performance under no attack |
| Mixtral-8x7B | Performance under no attack | 0.25%% | Delimiters#A4.T19; row Mixtral-8x7B; condition Delimiters; column Performance under no attack |
| Mixtral-8x7B | Change from no-defense PNA | 0.25%percentage points | Delimiters#A4.T19; row Mixtral-8x7B; condition Delimiters; column Change from no-defense PNA |
| Mixtral-8x7B | Performance under no attack | 0.00%% | Paraphrasing#A4.T19; row Mixtral-8x7B; condition Paraphrasing; column Performance under no attack |
| Mixtral-8x7B | Change from no-defense PNA | 0.00%percentage points | Paraphrasing#A4.T19; row Mixtral-8x7B; condition Paraphrasing; column Change from no-defense PNA |
| Mixtral-8x7B | Performance under no attack | 0.00%% | Instructional#A4.T19; row Mixtral-8x7B; condition Instructional; column Performance under no attack |
| Mixtral-8x7B | Change from no-defense PNA | 0.00%percentage points | Instructional#A4.T19; row Mixtral-8x7B; condition Instructional; column Change from no-defense PNA |
| Mixtral-8x7B | Performance under no attack | 0.25%% | Sandwich#A4.T19; row Mixtral-8x7B; condition Sandwich; column Performance under no attack |
| Mixtral-8x7B | Change from no-defense PNA | 0.25%percentage points | Sandwich#A4.T19; row Mixtral-8x7B; condition Sandwich; column Change from no-defense PNA |
| Mixtral-8x7B | Performance under no attack | 0.30%% | DPR#A4.T19; row Mixtral-8x7B; condition DPR; column Performance under no attack |
| Mixtral-8x7B | Change from no-defense PNA | 0.30%percentage points | DPR#A4.T19; row Mixtral-8x7B; condition DPR; column Change from no-defense PNA |
| Qwen2-7B | Performance under no attack | 9.75%% | No attack#A4.T19; row Qwen2-7B; condition No attack; column Performance under no attack |
| Qwen2-7B | Performance under no attack | 1.75%% | Delimiters#A4.T19; row Qwen2-7B; condition Delimiters; column Performance under no attack |
| Qwen2-7B | Change from no-defense PNA | -8.00%percentage points | Delimiters#A4.T19; row Qwen2-7B; condition Delimiters; column Change from no-defense PNA |
| Qwen2-7B | Performance under no attack | 7.00%% | Paraphrasing#A4.T19; row Qwen2-7B; condition Paraphrasing; column Performance under no attack |
| Qwen2-7B | Change from no-defense PNA | -2.75%percentage points | Paraphrasing#A4.T19; row Qwen2-7B; condition Paraphrasing; column Change from no-defense PNA |
| Qwen2-7B | Performance under no attack | 5.75%% | Instructional#A4.T19; row Qwen2-7B; condition Instructional; column Performance under no attack |
| Qwen2-7B | Change from no-defense PNA | -4.00%percentage points | Instructional#A4.T19; row Qwen2-7B; condition Instructional; column Change from no-defense PNA |
| Qwen2-7B | Performance under no attack | 5.25%% | Sandwich#A4.T19; row Qwen2-7B; condition Sandwich; column Performance under no attack |
| Qwen2-7B | Change from no-defense PNA | -4.50%percentage points | Sandwich#A4.T19; row Qwen2-7B; condition Sandwich; column Change from no-defense PNA |
| Qwen2-7B | Performance under no attack | 6.80%% | DPR#A4.T19; row Qwen2-7B; condition DPR; column Performance under no attack |
| Qwen2-7B | Change from no-defense PNA | -2.95%percentage points | DPR#A4.T19; row Qwen2-7B; condition DPR; column Change from no-defense PNA |
| Qwen2-72B | Performance under no attack | 4.00%% | No attack#A4.T19; row Qwen2-72B; condition No attack; column Performance under no attack |
| Qwen2-72B | Performance under no attack | 0.25%% | Delimiters#A4.T19; row Qwen2-72B; condition Delimiters; column Performance under no attack |
| Qwen2-72B | Change from no-defense PNA | -3.75%percentage points | Delimiters#A4.T19; row Qwen2-72B; condition Delimiters; column Change from no-defense PNA |
| Qwen2-72B | Performance under no attack | 0.00%% | Paraphrasing#A4.T19; row Qwen2-72B; condition Paraphrasing; column Performance under no attack |
| Qwen2-72B | Change from no-defense PNA | -4.00%percentage points | Paraphrasing#A4.T19; row Qwen2-72B; condition Paraphrasing; column Change from no-defense PNA |
| Qwen2-72B | Performance under no attack | 0.50%% | Instructional#A4.T19; row Qwen2-72B; condition Instructional; column Performance under no attack |
| Qwen2-72B | Change from no-defense PNA | -3.50%percentage points | Instructional#A4.T19; row Qwen2-72B; condition Instructional; column Change from no-defense PNA |
| Qwen2-72B | Performance under no attack | 0.50%% | Sandwich#A4.T19; row Qwen2-72B; condition Sandwich; column Performance under no attack |
| Qwen2-72B | Change from no-defense PNA | -3.50%percentage points | Sandwich#A4.T19; row Qwen2-72B; condition Sandwich; column Change from no-defense PNA |
| Qwen2-72B | Performance under no attack | 0.00%% | DPR#A4.T19; row Qwen2-72B; condition DPR; column Performance under no attack |
| Qwen2-72B | Change from no-defense PNA | -4.00%percentage points | DPR#A4.T19; row Qwen2-72B; condition DPR; column Change from no-defense PNA |
| Claude-3.5 Sonnet | Performance under no attack | 100.00%% | No attack#A4.T19; row Claude-3.5 Sonnet; condition No attack; column Performance under no attack |
| Claude-3.5 Sonnet | Performance under no attack | 90.00%% | Delimiters#A4.T19; row Claude-3.5 Sonnet; condition Delimiters; column Performance under no attack |
| Claude-3.5 Sonnet | Change from no-defense PNA | -10.00%percentage points | Delimiters#A4.T19; row Claude-3.5 Sonnet; condition Delimiters; column Change from no-defense PNA |
| Claude-3.5 Sonnet | Performance under no attack | 99.75%% | Paraphrasing#A4.T19; row Claude-3.5 Sonnet; condition Paraphrasing; column Performance under no attack |
| Claude-3.5 Sonnet | Change from no-defense PNA | -0.25%percentage points | Paraphrasing#A4.T19; row Claude-3.5 Sonnet; condition Paraphrasing; column Change from no-defense PNA |
| Claude-3.5 Sonnet | Performance under no attack | 90.00%% | Instructional#A4.T19; row Claude-3.5 Sonnet; condition Instructional; column Performance under no attack |
| Claude-3.5 Sonnet | Change from no-defense PNA | -10.00%percentage points | Instructional#A4.T19; row Claude-3.5 Sonnet; condition Instructional; column Change from no-defense PNA |
| Claude-3.5 Sonnet | Performance under no attack | 100.00%% | Sandwich#A4.T19; row Claude-3.5 Sonnet; condition Sandwich; column Performance under no attack |
| Claude-3.5 Sonnet | Change from no-defense PNA | 0.00%percentage points | Sandwich#A4.T19; row Claude-3.5 Sonnet; condition Sandwich; column Change from no-defense PNA |
| Claude-3.5 Sonnet | Performance under no attack | 95.80%% | DPR#A4.T19; row Claude-3.5 Sonnet; condition DPR; column Performance under no attack |
| Claude-3.5 Sonnet | Change from no-defense PNA | -4.20%percentage points | DPR#A4.T19; row Claude-3.5 Sonnet; condition DPR; column Change from no-defense PNA |
| GPT-3.5 Turbo | Performance under no attack | 8.00%% | No attack#A4.T19; row GPT-3.5 Turbo; condition No attack; column Performance under no attack |
| GPT-3.5 Turbo | Performance under no attack | 0.75%% | Delimiters#A4.T19; row GPT-3.5 Turbo; condition Delimiters; column Performance under no attack |
| GPT-3.5 Turbo | Change from no-defense PNA | -7.25%percentage points | Delimiters#A4.T19; row GPT-3.5 Turbo; condition Delimiters; column Change from no-defense PNA |
| GPT-3.5 Turbo | Performance under no attack | 6.75%% | Paraphrasing#A4.T19; row GPT-3.5 Turbo; condition Paraphrasing; column Performance under no attack |
| GPT-3.5 Turbo | Change from no-defense PNA | -1.25%percentage points | Paraphrasing#A4.T19; row GPT-3.5 Turbo; condition Paraphrasing; column Change from no-defense PNA |
| GPT-3.5 Turbo | Performance under no attack | 13.50%% | Instructional#A4.T19; row GPT-3.5 Turbo; condition Instructional; column Performance under no attack |
| GPT-3.5 Turbo | Change from no-defense PNA | 5.50%percentage points | Instructional#A4.T19; row GPT-3.5 Turbo; condition Instructional; column Change from no-defense PNA |
| GPT-3.5 Turbo | Performance under no attack | 10.25%% | Sandwich#A4.T19; row GPT-3.5 Turbo; condition Sandwich; column Performance under no attack |
| GPT-3.5 Turbo | Change from no-defense PNA | 2.25%percentage points | Sandwich#A4.T19; row GPT-3.5 Turbo; condition Sandwich; column Change from no-defense PNA |
| GPT-3.5 Turbo | Performance under no attack | 9.50%% | DPR#A4.T19; row GPT-3.5 Turbo; condition DPR; column Performance under no attack |
| GPT-3.5 Turbo | Change from no-defense PNA | 1.50%percentage points | DPR#A4.T19; row GPT-3.5 Turbo; condition DPR; column Change from no-defense PNA |
| GPT-4o | Performance under no attack | 79.00%% | No attack#A4.T19; row GPT-4o; condition No attack; column Performance under no attack |
| GPT-4o | Performance under no attack | 70.25%% | Delimiters#A4.T19; row GPT-4o; condition Delimiters; column Performance under no attack |
| GPT-4o | Change from no-defense PNA | -8.75%percentage points | Delimiters#A4.T19; row GPT-4o; condition Delimiters; column Change from no-defense PNA |
| GPT-4o | Performance under no attack | 80.00%% | Paraphrasing#A4.T19; row GPT-4o; condition Paraphrasing; column Performance under no attack |
| GPT-4o | Change from no-defense PNA | 1.00%percentage points | Paraphrasing#A4.T19; row GPT-4o; condition Paraphrasing; column Change from no-defense PNA |
| GPT-4o | Performance under no attack | 72.75%% | Instructional#A4.T19; row GPT-4o; condition Instructional; column Performance under no attack |
| GPT-4o | Change from no-defense PNA | -6.25%percentage points | Instructional#A4.T19; row GPT-4o; condition Instructional; column Change from no-defense PNA |
| GPT-4o | Performance under no attack | 79.50%% | Sandwich#A4.T19; row GPT-4o; condition Sandwich; column Performance under no attack |
| GPT-4o | Change from no-defense PNA | 0.50%percentage points | Sandwich#A4.T19; row GPT-4o; condition Sandwich; column Change from no-defense PNA |
| GPT-4o | Performance under no attack | 73.80%% | DPR#A4.T19; row GPT-4o; condition DPR; column Performance under no attack |
| GPT-4o | Change from no-defense PNA | -5.20%percentage points | DPR#A4.T19; row GPT-4o; condition DPR; column Change from no-defense PNA |
| GPT-4o-mini | Performance under no attack | 50.00%% | No attack#A4.T19; row GPT-4o-mini; condition No attack; column Performance under no attack |
| GPT-4o-mini | Performance under no attack | 37.00%% | Delimiters#A4.T19; row GPT-4o-mini; condition Delimiters; column Performance under no attack |
| GPT-4o-mini | Change from no-defense PNA | -13.00%percentage points | Delimiters#A4.T19; row GPT-4o-mini; condition Delimiters; column Change from no-defense PNA |
| GPT-4o-mini | Performance under no attack | 36.50%% | Paraphrasing#A4.T19; row GPT-4o-mini; condition Paraphrasing; column Performance under no attack |
| GPT-4o-mini | Change from no-defense PNA | -13.50%percentage points | Paraphrasing#A4.T19; row GPT-4o-mini; condition Paraphrasing; column Change from no-defense PNA |
| GPT-4o-mini | Performance under no attack | 42.50%% | Instructional#A4.T19; row GPT-4o-mini; condition Instructional; column Performance under no attack |
| GPT-4o-mini | Change from no-defense PNA | -7.50%percentage points | Instructional#A4.T19; row GPT-4o-mini; condition Instructional; column Change from no-defense PNA |
| GPT-4o-mini | Performance under no attack | 37.75%% | Sandwich#A4.T19; row GPT-4o-mini; condition Sandwich; column Performance under no attack |
| GPT-4o-mini | Change from no-defense PNA | -12.25%percentage points | Sandwich#A4.T19; row GPT-4o-mini; condition Sandwich; column Change from no-defense PNA |
| GPT-4o-mini | Performance under no attack | 29.70%% | DPR#A4.T19; row GPT-4o-mini; condition DPR; column Performance under no attack |
| GPT-4o-mini | Change from no-defense PNA | -20.30%percentage points | DPR#A4.T19; row GPT-4o-mini; condition DPR; column Change from no-defense PNA |
| Average | Performance under no attack | 29.46%% | No attack#A4.T19; row Average; condition No attack; column Performance under no attack |
| Average | Performance under no attack | 22.52%% | Delimiters#A4.T19; row Average; condition Delimiters; column Performance under no attack |
| Average | Change from no-defense PNA | -6.94%percentage points | Delimiters#A4.T19; row Average; condition Delimiters; column Change from no-defense PNA |
| Average | Performance under no attack | 27.69%% | Paraphrasing#A4.T19; row Average; condition Paraphrasing; column Performance under no attack |
| Average | Change from no-defense PNA | -1.77%percentage points | Paraphrasing#A4.T19; row Average; condition Paraphrasing; column Change from no-defense PNA |
| Average | Performance under no attack | 26.63%% | Instructional#A4.T19; row Average; condition Instructional; column Performance under no attack |
| Average | Change from no-defense PNA | -2.83%percentage points | Instructional#A4.T19; row Average; condition Instructional; column Change from no-defense PNA |
| Average | Performance under no attack | 28.29%% | Sandwich#A4.T19; row Average; condition Sandwich; column Performance under no attack |
| Average | Change from no-defense PNA | -1.17%percentage points | Sandwich#A4.T19; row Average; condition Sandwich; column Change from no-defense PNA |
| Average | Performance under no attack | 26.38%% | DPR#A4.T19; row Average; condition DPR; column Performance under no attack |
| Average | Change from no-defense PNA | -3.08%percentage points | DPR#A4.T19; row Average; condition DPR; column Change from no-defense PNA |
| Gemma2-9B | Attack success rate | 39.75%% | PoT baseline#A4.T20; row Gemma2-9B; condition PoT baseline; column Attack success rate |
| Gemma2-9B | Performance under no attack | 10.75%% | PoT baseline#A4.T20; row Gemma2-9B; condition PoT baseline; column Performance under no attack |
| Gemma2-9B | Attack success rate | 67.25%% | Shuffle defense#A4.T20; row Gemma2-9B; condition Shuffle defense; column Attack success rate |
| Gemma2-9B | Performance under no attack | 22.25%% | Shuffle defense#A4.T20; row Gemma2-9B; condition Shuffle defense; column Performance under no attack |
| Gemma2-9B | Attack success rate | 24.50%% | Paraphrase defense#A4.T20; row Gemma2-9B; condition Paraphrase defense; column Attack success rate |
| Gemma2-9B | Performance under no attack | 21.75%% | Paraphrase defense#A4.T20; row Gemma2-9B; condition Paraphrase defense; column Performance under no attack |
| Gemma2-27B | Attack success rate | 54.50%% | PoT baseline#A4.T20; row Gemma2-27B; condition PoT baseline; column Attack success rate |
| Gemma2-27B | Performance under no attack | 31.50%% | PoT baseline#A4.T20; row Gemma2-27B; condition PoT baseline; column Performance under no attack |
| Gemma2-27B | Attack success rate | 59.50%% | Shuffle defense#A4.T20; row Gemma2-27B; condition Shuffle defense; column Attack success rate |
| Gemma2-27B | Performance under no attack | 40.75%% | Shuffle defense#A4.T20; row Gemma2-27B; condition Shuffle defense; column Performance under no attack |
| Gemma2-27B | Attack success rate | 23.25%% | Paraphrase defense#A4.T20; row Gemma2-27B; condition Paraphrase defense; column Attack success rate |
| Gemma2-27B | Performance under no attack | 32.25%% | Paraphrase defense#A4.T20; row Gemma2-27B; condition Paraphrase defense; column Performance under no attack |
| LLaMA3-8B | Attack success rate | 21.50%% | PoT baseline#A4.T20; row LLaMA3-8B; condition PoT baseline; column Attack success rate |
| LLaMA3-8B | Performance under no attack | 1.50%% | PoT baseline#A4.T20; row LLaMA3-8B; condition PoT baseline; column Performance under no attack |
| LLaMA3-8B | Attack success rate | 2.25%% | Shuffle defense#A4.T20; row LLaMA3-8B; condition Shuffle defense; column Attack success rate |
| LLaMA3-8B | Performance under no attack | 3.50%% | Shuffle defense#A4.T20; row LLaMA3-8B; condition Shuffle defense; column Performance under no attack |
| LLaMA3-8B | Attack success rate | 5.00%% | Paraphrase defense#A4.T20; row LLaMA3-8B; condition Paraphrase defense; column Attack success rate |
| LLaMA3-8B | Performance under no attack | 6.00%% | Paraphrase defense#A4.T20; row LLaMA3-8B; condition Paraphrase defense; column Performance under no attack |
| LLaMA3-70B | Attack success rate | 57.00%% | PoT baseline#A4.T20; row LLaMA3-70B; condition PoT baseline; column Attack success rate |
| LLaMA3-70B | Performance under no attack | 66.50%% | PoT baseline#A4.T20; row LLaMA3-70B; condition PoT baseline; column Performance under no attack |
| LLaMA3-70B | Attack success rate | 63.75%% | Shuffle defense#A4.T20; row LLaMA3-70B; condition Shuffle defense; column Attack success rate |
| LLaMA3-70B | Performance under no attack | 54.50%% | Shuffle defense#A4.T20; row LLaMA3-70B; condition Shuffle defense; column Performance under no attack |
| LLaMA3-70B | Attack success rate | 44.75%% | Paraphrase defense#A4.T20; row LLaMA3-70B; condition Paraphrase defense; column Attack success rate |
| LLaMA3-70B | Performance under no attack | 52.75%% | Paraphrase defense#A4.T20; row LLaMA3-70B; condition Paraphrase defense; column Performance under no attack |
| LLaMA3.1-8B | Attack success rate | 19.00%% | PoT baseline#A4.T20; row LLaMA3.1-8B; condition PoT baseline; column Attack success rate |
| LLaMA3.1-8B | Performance under no attack | 0.75%% | PoT baseline#A4.T20; row LLaMA3.1-8B; condition PoT baseline; column Performance under no attack |
| LLaMA3.1-8B | Attack success rate | 17.25%% | Shuffle defense#A4.T20; row LLaMA3.1-8B; condition Shuffle defense; column Attack success rate |
| LLaMA3.1-8B | Performance under no attack | 2.75%% | Shuffle defense#A4.T20; row LLaMA3.1-8B; condition Shuffle defense; column Performance under no attack |
| LLaMA3.1-8B | Attack success rate | 17.50%% | Paraphrase defense#A4.T20; row LLaMA3.1-8B; condition Paraphrase defense; column Attack success rate |
| LLaMA3.1-8B | Performance under no attack | 2.50%% | Paraphrase defense#A4.T20; row LLaMA3.1-8B; condition Paraphrase defense; column Performance under no attack |
| LLaMA3.1-70B | Attack success rate | 59.75%% | PoT baseline#A4.T20; row LLaMA3.1-70B; condition PoT baseline; column Attack success rate |
| LLaMA3.1-70B | Performance under no attack | 21.25%% | PoT baseline#A4.T20; row LLaMA3.1-70B; condition PoT baseline; column Performance under no attack |
| LLaMA3.1-70B | Attack success rate | 69.00%% | Shuffle defense#A4.T20; row LLaMA3.1-70B; condition Shuffle defense; column Attack success rate |
| LLaMA3.1-70B | Performance under no attack | 43.00%% | Shuffle defense#A4.T20; row LLaMA3.1-70B; condition Shuffle defense; column Performance under no attack |
| LLaMA3.1-70B | Attack success rate | 42.00%% | Paraphrase defense#A4.T20; row LLaMA3.1-70B; condition Paraphrase defense; column Attack success rate |
| LLaMA3.1-70B | Performance under no attack | 30.00%% | Paraphrase defense#A4.T20; row LLaMA3.1-70B; condition Paraphrase defense; column Performance under no attack |
| Mixtral-8x7B | Attack success rate | 4.75%% | PoT baseline#A4.T20; row Mixtral-8x7B; condition PoT baseline; column Attack success rate |
| Mixtral-8x7B | Performance under no attack | 0.00%% | PoT baseline#A4.T20; row Mixtral-8x7B; condition PoT baseline; column Performance under no attack |
| Mixtral-8x7B | Attack success rate | 12.25%% | Shuffle defense#A4.T20; row Mixtral-8x7B; condition Shuffle defense; column Attack success rate |
| Mixtral-8x7B | Performance under no attack | 0.25%% | Shuffle defense#A4.T20; row Mixtral-8x7B; condition Shuffle defense; column Performance under no attack |
| Mixtral-8x7B | Attack success rate | 4.50%% | Paraphrase defense#A4.T20; row Mixtral-8x7B; condition Paraphrase defense; column Attack success rate |
| Mixtral-8x7B | Performance under no attack | 0.50%% | Paraphrase defense#A4.T20; row Mixtral-8x7B; condition Paraphrase defense; column Performance under no attack |
| Qwen2-7B | Attack success rate | 12.25%% | PoT baseline#A4.T20; row Qwen2-7B; condition PoT baseline; column Attack success rate |
| Qwen2-7B | Performance under no attack | 9.75%% | PoT baseline#A4.T20; row Qwen2-7B; condition PoT baseline; column Performance under no attack |
| Qwen2-7B | Attack success rate | 14.50%% | Shuffle defense#A4.T20; row Qwen2-7B; condition Shuffle defense; column Attack success rate |
| Qwen2-7B | Performance under no attack | 13.00%% | Shuffle defense#A4.T20; row Qwen2-7B; condition Shuffle defense; column Performance under no attack |
| Qwen2-7B | Attack success rate | 11.00%% | Paraphrase defense#A4.T20; row Qwen2-7B; condition Paraphrase defense; column Attack success rate |
| Qwen2-7B | Performance under no attack | 10.25%% | Paraphrase defense#A4.T20; row Qwen2-7B; condition Paraphrase defense; column Performance under no attack |
| Qwen2-72B | Attack success rate | 57.75%% | PoT baseline#A4.T20; row Qwen2-72B; condition PoT baseline; column Attack success rate |
| Qwen2-72B | Performance under no attack | 4.00%% | PoT baseline#A4.T20; row Qwen2-72B; condition PoT baseline; column Performance under no attack |
| Qwen2-72B | Attack success rate | 22.75%% | Shuffle defense#A4.T20; row Qwen2-72B; condition Shuffle defense; column Attack success rate |
| Qwen2-72B | Performance under no attack | 10.75%% | Shuffle defense#A4.T20; row Qwen2-72B; condition Shuffle defense; column Performance under no attack |
| Qwen2-72B | Attack success rate | 37.75%% | Paraphrase defense#A4.T20; row Qwen2-72B; condition Paraphrase defense; column Attack success rate |
| Qwen2-72B | Performance under no attack | 18.00%% | Paraphrase defense#A4.T20; row Qwen2-72B; condition Paraphrase defense; column Performance under no attack |
| Claude-3.5 Sonnet | Attack success rate | 17.50%% | PoT baseline#A4.T20; row Claude-3.5 Sonnet; condition PoT baseline; column Attack success rate |
| Claude-3.5 Sonnet | Performance under no attack | 100.00%% | PoT baseline#A4.T20; row Claude-3.5 Sonnet; condition PoT baseline; column Performance under no attack |
| Claude-3.5 Sonnet | Attack success rate | 93.50%% | Shuffle defense#A4.T20; row Claude-3.5 Sonnet; condition Shuffle defense; column Attack success rate |
| Claude-3.5 Sonnet | Performance under no attack | 81.50%% | Shuffle defense#A4.T20; row Claude-3.5 Sonnet; condition Shuffle defense; column Performance under no attack |
| Claude-3.5 Sonnet | Attack success rate | 13.75%% | Paraphrase defense#A4.T20; row Claude-3.5 Sonnet; condition Paraphrase defense; column Attack success rate |
| Claude-3.5 Sonnet | Performance under no attack | 82.75%% | Paraphrase defense#A4.T20; row Claude-3.5 Sonnet; condition Paraphrase defense; column Performance under no attack |
| GPT-3.5 Turbo | Attack success rate | 8.25%% | PoT baseline#A4.T20; row GPT-3.5 Turbo; condition PoT baseline; column Attack success rate |
| GPT-3.5 Turbo | Performance under no attack | 8.00%% | PoT baseline#A4.T20; row GPT-3.5 Turbo; condition PoT baseline; column Performance under no attack |
| GPT-3.5 Turbo | Attack success rate | 16.50%% | Shuffle defense#A4.T20; row GPT-3.5 Turbo; condition Shuffle defense; column Attack success rate |
| GPT-3.5 Turbo | Performance under no attack | 16.75%% | Shuffle defense#A4.T20; row GPT-3.5 Turbo; condition Shuffle defense; column Performance under no attack |
| GPT-3.5 Turbo | Attack success rate | 6.25%% | Paraphrase defense#A4.T20; row GPT-3.5 Turbo; condition Paraphrase defense; column Attack success rate |
| GPT-3.5 Turbo | Performance under no attack | 23.50%% | Paraphrase defense#A4.T20; row GPT-3.5 Turbo; condition Paraphrase defense; column Performance under no attack |
| GPT-4o | Attack success rate | 100.00%% | PoT baseline#A4.T20; row GPT-4o; condition PoT baseline; column Attack success rate |
| GPT-4o | Performance under no attack | 79.00%% | PoT baseline#A4.T20; row GPT-4o; condition PoT baseline; column Performance under no attack |
| GPT-4o | Attack success rate | 98.50%% | Shuffle defense#A4.T20; row GPT-4o; condition Shuffle defense; column Attack success rate |
| GPT-4o | Performance under no attack | 78.50%% | Shuffle defense#A4.T20; row GPT-4o; condition Shuffle defense; column Performance under no attack |
| GPT-4o | Attack success rate | 84.75%% | Paraphrase defense#A4.T20; row GPT-4o; condition Paraphrase defense; column Attack success rate |
| GPT-4o | Performance under no attack | 88.00%% | Paraphrase defense#A4.T20; row GPT-4o; condition Paraphrase defense; column Performance under no attack |
| GPT-4o-mini | Attack success rate | 95.50%% | PoT baseline#A4.T20; row GPT-4o-mini; condition PoT baseline; column Attack success rate |
| GPT-4o-mini | Performance under no attack | 50.00%% | PoT baseline#A4.T20; row GPT-4o-mini; condition PoT baseline; column Performance under no attack |
| GPT-4o-mini | Attack success rate | 39.75%% | Shuffle defense#A4.T20; row GPT-4o-mini; condition Shuffle defense; column Attack success rate |
| GPT-4o-mini | Performance under no attack | 63.75%% | Shuffle defense#A4.T20; row GPT-4o-mini; condition Shuffle defense; column Performance under no attack |
| GPT-4o-mini | Attack success rate | 62.75%% | Paraphrase defense#A4.T20; row GPT-4o-mini; condition Paraphrase defense; column Attack success rate |
| GPT-4o-mini | Performance under no attack | 79.00%% | Paraphrase defense#A4.T20; row GPT-4o-mini; condition Paraphrase defense; column Performance under no attack |
| Average | Attack success rate | 42.12%% | PoT baseline#A4.T20; row Average; condition PoT baseline; column Attack success rate |
| Average | Performance under no attack | 29.46%% | PoT baseline#A4.T20; row Average; condition PoT baseline; column Performance under no attack |
| Average | Attack success rate | 44.37%% | Shuffle defense#A4.T20; row Average; condition Shuffle defense; column Attack success rate |
| Average | Performance under no attack | 33.17%% | Shuffle defense#A4.T20; row Average; condition Shuffle defense; column Performance under no attack |
| Average | Attack success rate | 29.06%% | Paraphrase defense#A4.T20; row Average; condition Paraphrase defense; column Attack success rate |
| Average | Performance under no attack | 34.40%% | Paraphrase defense#A4.T20; row Average; condition Paraphrase defense; column Performance under no attack |
| Gemma2-9B | False negative rate | 0.658ratio | Memory attack detection#A4.T21; row Gemma2-9B; condition Memory attack detection; column False negative rate |
| Gemma2-9B | False positive rate | 0.204ratio | Memory attack detection#A4.T21; row Gemma2-9B; condition Memory attack detection; column False positive rate |
| Gemma2-27B | False negative rate | 0.655ratio | Memory attack detection#A4.T21; row Gemma2-27B; condition Memory attack detection; column False negative rate |
| Gemma2-27B | False positive rate | 0.201ratio | Memory attack detection#A4.T21; row Gemma2-27B; condition Memory attack detection; column False positive rate |
| LLaMA3-8B | False negative rate | 0.654ratio | Memory attack detection#A4.T21; row LLaMA3-8B; condition Memory attack detection; column False negative rate |
| LLaMA3-8B | False positive rate | 0.204ratio | Memory attack detection#A4.T21; row LLaMA3-8B; condition Memory attack detection; column False positive rate |
| LLaMA3-70B | False negative rate | 0.661ratio | Memory attack detection#A4.T21; row LLaMA3-70B; condition Memory attack detection; column False negative rate |
| LLaMA3-70B | False positive rate | 0.202ratio | Memory attack detection#A4.T21; row LLaMA3-70B; condition Memory attack detection; column False positive rate |
| LLaMA3.1-8B | False negative rate | 0.656ratio | Memory attack detection#A4.T21; row LLaMA3.1-8B; condition Memory attack detection; column False negative rate |
| LLaMA3.1-8B | False positive rate | 0.200ratio | Memory attack detection#A4.T21; row LLaMA3.1-8B; condition Memory attack detection; column False positive rate |
| LLaMA3.1-70B | False negative rate | 0.659ratio | Memory attack detection#A4.T21; row LLaMA3.1-70B; condition Memory attack detection; column False negative rate |
| LLaMA3.1-70B | False positive rate | 0.197ratio | Memory attack detection#A4.T21; row LLaMA3.1-70B; condition Memory attack detection; column False positive rate |
| Mixtral-8x7B | False negative rate | 0.665ratio | Memory attack detection#A4.T21; row Mixtral-8x7B; condition Memory attack detection; column False negative rate |
| Mixtral-8x7B | False positive rate | 0.203ratio | Memory attack detection#A4.T21; row Mixtral-8x7B; condition Memory attack detection; column False positive rate |
| Qwen2-7B | False negative rate | 0.657ratio | Memory attack detection#A4.T21; row Qwen2-7B; condition Memory attack detection; column False negative rate |
| Qwen2-7B | False positive rate | 0.193ratio | Memory attack detection#A4.T21; row Qwen2-7B; condition Memory attack detection; column False positive rate |
| Qwen2-72B | False negative rate | 0.671ratio | Memory attack detection#A4.T21; row Qwen2-72B; condition Memory attack detection; column False negative rate |
| Qwen2-72B | False positive rate | 0.198ratio | Memory attack detection#A4.T21; row Qwen2-72B; condition Memory attack detection; column False positive rate |
| Claude-3.5 Sonnet | False negative rate | 0.663ratio | Memory attack detection#A4.T21; row Claude-3.5 Sonnet; condition Memory attack detection; column False negative rate |
| Claude-3.5 Sonnet | False positive rate | 0.199ratio | Memory attack detection#A4.T21; row Claude-3.5 Sonnet; condition Memory attack detection; column False positive rate |
| GPT-3.5 Turbo | False negative rate | 0.661ratio | Memory attack detection#A4.T21; row GPT-3.5 Turbo; condition Memory attack detection; column False negative rate |
| GPT-3.5 Turbo | False positive rate | 0.198ratio | Memory attack detection#A4.T21; row GPT-3.5 Turbo; condition Memory attack detection; column False positive rate |
| GPT-4o | False negative rate | 0.664ratio | Memory attack detection#A4.T21; row GPT-4o; condition Memory attack detection; column False negative rate |
| GPT-4o | False positive rate | 0.203ratio | Memory attack detection#A4.T21; row GPT-4o; condition Memory attack detection; column False positive rate |
| GPT-4o-mini | False negative rate | 0.657ratio | Memory attack detection#A4.T21; row GPT-4o-mini; condition Memory attack detection; column False negative rate |
| GPT-4o-mini | False positive rate | 0.200ratio | Memory attack detection#A4.T21; row GPT-4o-mini; condition Memory attack detection; column False positive rate |
| Average | False negative rate | 0.660ratio | Memory attack detection#A4.T21; row Average; condition Memory attack detection; column False negative rate |
| Average | False positive rate | 0.200ratio | Memory attack detection#A4.T21; row Average; condition Memory attack detection; column False positive rate |
| Aggregate across evaluated backbones | Attack success rate | 78.38%% | Combined Attack / DPI#A4.T17; row Aggregate across evaluated backbones; condition Combined Attack / DPI; column Attack success rate |
| Aggregate across evaluated backbones | Refusal rate | 5.35%% | Combined Attack / DPI#A4.T17; row Aggregate across evaluated backbones; condition Combined Attack / DPI; column Refusal rate |
| Aggregate across evaluated backbones | Attack success rate | 27.98%% | Combined Attack / IPI#A4.T17; row Aggregate across evaluated backbones; condition Combined Attack / IPI; column Attack success rate |
| Aggregate across evaluated backbones | Refusal rate | 10.27%% | Combined Attack / IPI#A4.T17; row Aggregate across evaluated backbones; condition Combined Attack / IPI; column Refusal rate |
| Aggregate across evaluated backbones | Attack success rate | 7.65%% | Combined Attack / Memory poisoning#A4.T17; row Aggregate across evaluated backbones; condition Combined Attack / Memory poisoning; column Attack success rate |
| Aggregate across evaluated backbones | Refusal rate | 4.75%% | Combined Attack / Memory poisoning#A4.T17; row Aggregate across evaluated backbones; condition Combined Attack / Memory poisoning; column Refusal rate |
| Aggregate across evaluated backbones | Attack success rate | 38.01%% | Combined Attack / Average#A4.T17; row Aggregate across evaluated backbones; condition Combined Attack / Average; column Attack success rate |
| Aggregate across evaluated backbones | Refusal rate | 6.79%% | Combined Attack / Average#A4.T17; row Aggregate across evaluated backbones; condition Combined Attack / Average; column Refusal rate |
| Aggregate across evaluated backbones | Attack success rate | 73.85%% | Context Ignoring / DPI#A4.T17; row Aggregate across evaluated backbones; condition Context Ignoring / DPI; column Attack success rate |
| Aggregate across evaluated backbones | Refusal rate | 6.33%% | Context Ignoring / DPI#A4.T17; row Aggregate across evaluated backbones; condition Context Ignoring / DPI; column Refusal rate |
| Aggregate across evaluated backbones | Attack success rate | 27.29%% | Context Ignoring / IPI#A4.T17; row Aggregate across evaluated backbones; condition Context Ignoring / IPI; column Attack success rate |
| Aggregate across evaluated backbones | Refusal rate | 9.50%% | Context Ignoring / IPI#A4.T17; row Aggregate across evaluated backbones; condition Context Ignoring / IPI; column Refusal rate |
| Aggregate across evaluated backbones | Attack success rate | 8.52%% | Context Ignoring / Memory poisoning#A4.T17; row Aggregate across evaluated backbones; condition Context Ignoring / Memory poisoning; column Attack success rate |
| Aggregate across evaluated backbones | Refusal rate | 4.58%% | Context Ignoring / Memory poisoning#A4.T17; row Aggregate across evaluated backbones; condition Context Ignoring / Memory poisoning; column Refusal rate |
| Aggregate across evaluated backbones | Attack success rate | 36.55%% | Context Ignoring / Average#A4.T17; row Aggregate across evaluated backbones; condition Context Ignoring / Average; column Attack success rate |
| Aggregate across evaluated backbones | Refusal rate | 6.80%% | Context Ignoring / Average#A4.T17; row Aggregate across evaluated backbones; condition Context Ignoring / Average; column Refusal rate |
| Aggregate across evaluated backbones | Attack success rate | 68.73%% | Escape Characters / DPI#A4.T17; row Aggregate across evaluated backbones; condition Escape Characters / DPI; column Attack success rate |
| Aggregate across evaluated backbones | Refusal rate | 7.21%% | Escape Characters / DPI#A4.T17; row Aggregate across evaluated backbones; condition Escape Characters / DPI; column Refusal rate |
| Aggregate across evaluated backbones | Attack success rate | 27.81%% | Escape Characters / IPI#A4.T17; row Aggregate across evaluated backbones; condition Escape Characters / IPI; column Attack success rate |
| Aggregate across evaluated backbones | Refusal rate | 7.62%% | Escape Characters / IPI#A4.T17; row Aggregate across evaluated backbones; condition Escape Characters / IPI; column Refusal rate |
| Aggregate across evaluated backbones | Attack success rate | 7.71%% | Escape Characters / Memory poisoning#A4.T17; row Aggregate across evaluated backbones; condition Escape Characters / Memory poisoning; column Attack success rate |
| Aggregate across evaluated backbones | Refusal rate | 4.38%% | Escape Characters / Memory poisoning#A4.T17; row Aggregate across evaluated backbones; condition Escape Characters / Memory poisoning; column Refusal rate |
| Aggregate across evaluated backbones | Attack success rate | 34.75%% | Escape Characters / Average#A4.T17; row Aggregate across evaluated backbones; condition Escape Characters / Average; column Attack success rate |
| Aggregate across evaluated backbones | Refusal rate | 6.40%% | Escape Characters / Average#A4.T17; row Aggregate across evaluated backbones; condition Escape Characters / Average; column Refusal rate |
| Aggregate across evaluated backbones | Attack success rate | 71.94%% | Fake Completion / DPI#A4.T17; row Aggregate across evaluated backbones; condition Fake Completion / DPI; column Attack success rate |
| Aggregate across evaluated backbones | Refusal rate | 5.63%% | Fake Completion / DPI#A4.T17; row Aggregate across evaluated backbones; condition Fake Completion / DPI; column Refusal rate |
| Aggregate across evaluated backbones | Attack success rate | 26.62%% | Fake Completion / IPI#A4.T17; row Aggregate across evaluated backbones; condition Fake Completion / IPI; column Attack success rate |
| Aggregate across evaluated backbones | Refusal rate | 8.19%% | Fake Completion / IPI#A4.T17; row Aggregate across evaluated backbones; condition Fake Completion / IPI; column Refusal rate |
| Aggregate across evaluated backbones | Attack success rate | 8.00%% | Fake Completion / Memory poisoning#A4.T17; row Aggregate across evaluated backbones; condition Fake Completion / Memory poisoning; column Attack success rate |
| Aggregate across evaluated backbones | Refusal rate | 4.87%% | Fake Completion / Memory poisoning#A4.T17; row Aggregate across evaluated backbones; condition Fake Completion / Memory poisoning; column Refusal rate |
| Aggregate across evaluated backbones | Attack success rate | 35.52%% | Fake Completion / Average#A4.T17; row Aggregate across evaluated backbones; condition Fake Completion / Average; column Attack success rate |
| Aggregate across evaluated backbones | Refusal rate | 6.23%% | Fake Completion / Average#A4.T17; row Aggregate across evaluated backbones; condition Fake Completion / Average; column Refusal rate |
| Aggregate across evaluated backbones | Attack success rate | 70.52%% | Naive / DPI#A4.T17; row Aggregate across evaluated backbones; condition Naive / DPI; column Attack success rate |
| Aggregate across evaluated backbones | Refusal rate | 8.15%% | Naive / DPI#A4.T17; row Aggregate across evaluated backbones; condition Naive / DPI; column Refusal rate |
| Aggregate across evaluated backbones | Attack success rate | 28.04%% | Naive / IPI#A4.T17; row Aggregate across evaluated backbones; condition Naive / IPI; column Attack success rate |
| Aggregate across evaluated backbones | Refusal rate | 7.46%% | Naive / IPI#A4.T17; row Aggregate across evaluated backbones; condition Naive / IPI; column Refusal rate |
| Aggregate across evaluated backbones | Attack success rate | 7.73%% | Naive / Memory poisoning#A4.T17; row Aggregate across evaluated backbones; condition Naive / Memory poisoning; column Attack success rate |
| Aggregate across evaluated backbones | Refusal rate | 4.56%% | Naive / Memory poisoning#A4.T17; row Aggregate across evaluated backbones; condition Naive / Memory poisoning; column Refusal rate |
| Aggregate across evaluated backbones | Attack success rate | 35.43%% | Naive / Average#A4.T17; row Aggregate across evaluated backbones; condition Naive / Average; column Attack success rate |
| Aggregate across evaluated backbones | Refusal rate | 6.72%% | Naive / Average#A4.T17; row Aggregate across evaluated backbones; condition Naive / Average; column Refusal rate |
| Aggregate across evaluated backbones | Attack success rate | 72.68%% | Average / DPI#A4.T17; row Aggregate across evaluated backbones; condition Average / DPI; column Attack success rate |
| Aggregate across evaluated backbones | Refusal rate | 6.53%% | Average / DPI#A4.T17; row Aggregate across evaluated backbones; condition Average / DPI; column Refusal rate |
| Aggregate across evaluated backbones | Attack success rate | 27.55%% | Average / IPI#A4.T17; row Aggregate across evaluated backbones; condition Average / IPI; column Attack success rate |
| Aggregate across evaluated backbones | Refusal rate | 8.61%% | Average / IPI#A4.T17; row Aggregate across evaluated backbones; condition Average / IPI; column Refusal rate |
| Aggregate across evaluated backbones | Attack success rate | 7.92%% | Average / Memory poisoning#A4.T17; row Aggregate across evaluated backbones; condition Average / Memory poisoning; column Attack success rate |
| Aggregate across evaluated backbones | Refusal rate | 4.63%% | Average / Memory poisoning#A4.T17; row Aggregate across evaluated backbones; condition Average / Memory poisoning; column Refusal rate |
| Aggregate across evaluated backbones | Attack success rate | 36.05%% | Average / Average#A4.T17; row Aggregate across evaluated backbones; condition Average / Average; column Attack success rate |
| Aggregate across evaluated backbones | Refusal rate | 6.59%% | Average / Average#A4.T17; row Aggregate across evaluated backbones; condition Average / Average; column Refusal rate |
| Aggregate across evaluated backbones | Attack success rate | 74.59%% | No / DPI#A4.T18; row Aggregate across evaluated backbones; condition No / DPI; column Attack success rate |
| Aggregate across evaluated backbones | Refusal rate | 3.51%% | No / DPI#A4.T18; row Aggregate across evaluated backbones; condition No / DPI; column Refusal rate |
| Aggregate across evaluated backbones | Attack success rate | 32.67%% | No / IPI#A4.T18; row Aggregate across evaluated backbones; condition No / IPI; column Attack success rate |
| Aggregate across evaluated backbones | Refusal rate | 6.52%% | No / IPI#A4.T18; row Aggregate across evaluated backbones; condition No / IPI; column Refusal rate |
| Aggregate across evaluated backbones | Attack success rate | 9.68%% | No / Memory poisoning#A4.T18; row Aggregate across evaluated backbones; condition No / Memory poisoning; column Attack success rate |
| Aggregate across evaluated backbones | Refusal rate | 4.57%% | No / Memory poisoning#A4.T18; row Aggregate across evaluated backbones; condition No / Memory poisoning; column Refusal rate |
| Aggregate across evaluated backbones | Attack success rate | 38.98%% | No / Average#A4.T18; row Aggregate across evaluated backbones; condition No / Average; column Attack success rate |
| Aggregate across evaluated backbones | Refusal rate | 4.87%% | No / Average#A4.T18; row Aggregate across evaluated backbones; condition No / Average; column Refusal rate |
| Aggregate across evaluated backbones | Attack success rate | 70.78%% | Yes / DPI#A4.T18; row Aggregate across evaluated backbones; condition Yes / DPI; column Attack success rate |
| Aggregate across evaluated backbones | Refusal rate | 9.56%% | Yes / DPI#A4.T18; row Aggregate across evaluated backbones; condition Yes / DPI; column Refusal rate |
| Aggregate across evaluated backbones | Attack success rate | 22.42%% | Yes / IPI#A4.T18; row Aggregate across evaluated backbones; condition Yes / IPI; column Attack success rate |
| Aggregate across evaluated backbones | Refusal rate | 10.69%% | Yes / IPI#A4.T18; row Aggregate across evaluated backbones; condition Yes / IPI; column Refusal rate |
| Aggregate across evaluated backbones | Attack success rate | 6.16%% | Yes / Memory poisoning#A4.T18; row Aggregate across evaluated backbones; condition Yes / Memory poisoning; column Attack success rate |
| Aggregate across evaluated backbones | Refusal rate | 4.68%% | Yes / Memory poisoning#A4.T18; row Aggregate across evaluated backbones; condition Yes / Memory poisoning; column Refusal rate |
| Aggregate across evaluated backbones | Attack success rate | 33.12%% | Yes / Average#A4.T18; row Aggregate across evaluated backbones; condition Yes / Average; column Attack success rate |
| Aggregate across evaluated backbones | Refusal rate | 8.31%% | Yes / Average#A4.T18; row Aggregate across evaluated backbones; condition Yes / Average; column Refusal rate |
| Aggregate across evaluated backbones | Attack success rate | 72.68%% | Average / DPI#A4.T18; row Aggregate across evaluated backbones; condition Average / DPI; column Attack success rate |
| Aggregate across evaluated backbones | Refusal rate | 6.53%% | Average / DPI#A4.T18; row Aggregate across evaluated backbones; condition Average / DPI; column Refusal rate |
| Aggregate across evaluated backbones | Attack success rate | 27.55%% | Average / IPI#A4.T18; row Aggregate across evaluated backbones; condition Average / IPI; column Attack success rate |
| Aggregate across evaluated backbones | Refusal rate | 8.61%% | Average / IPI#A4.T18; row Aggregate across evaluated backbones; condition Average / IPI; column Refusal rate |
| Aggregate across evaluated backbones | Attack success rate | 7.92%% | Average / Memory poisoning#A4.T18; row Aggregate across evaluated backbones; condition Average / Memory poisoning; column Attack success rate |
| Aggregate across evaluated backbones | Refusal rate | 4.63%% | Average / Memory poisoning#A4.T18; row Aggregate across evaluated backbones; condition Average / Memory poisoning; column Refusal rate |
| Aggregate across evaluated backbones | Attack success rate | 36.05%% | Average / Average#A4.T18; row Aggregate across evaluated backbones; condition Average / Average; column Attack success rate |
| Aggregate across evaluated backbones | Refusal rate | 6.59%% | Average / Average#A4.T18; row Aggregate across evaluated backbones; condition Average / Average; column Refusal rate |
阅读范围与复核
Selected original HTML sections and table context; not a complete paper/appendix audit or independent reproduction.
模型辅助整理，独立证据复核待完成。
核对原始来源 ↗