风险评测 · 评测基准

A StrongREJECT for Empty Jailbreaks

NUWA 研究解读 · 独立复核待完成

首次发表: 2024-02-15来源核验: 2026-10-25

原始来源 ↗引用本记录 ↓

NUWA Research Note

研究问题研究设计与假设来源发现 NUWA 解读局限与相反证据研究与决策意义机构与角色原始资源

模型辅助整理 · 选读原始全文或官方文章中的相关章节；尚未完成逐章节审计。

研究速读

NUWA 解读：表面攻击提升可能来自宽松判分，而非保留有害任务能力。

研究问题

越狱评估器测量的是有用有害内容，还是仅回答意愿？

研究设计与假设

精选禁答问题与量表、微调判分器对照人工判断，强调回答的具体性和有用性。

StrongREJECT — Figure 2 — Figure 2 · 原文结果图，请结合下方的协议范围与局限解读。原图与来源 ↗

来源发现

既有评估器经常高估越狱效果。

所报稳健性组合中，与人类攻击排序的相关均超过0.84（附录D）。

NUWA 解读

NUWA 解读：表面攻击提升可能来自宽松判分，而非保留有害任务能力。

局限与相反证据

部分模型人工标注攻击较少，限制相关估计；尚未验证多模态判分。

研究与决策意义

模型建议：采用攻击成功结论前，先以人工评分输出验证评估器。

精确来源位置

A StrongREJECT ¯ \overline{\hbox{{StrongREJECT}}} for Empty Jailbreaks · §2 The StrongREJECT benchmarkSelected original resource https://arxiv.org/html/2402.10260v2; sha256:ffbcbe49cc39b51c0cd776d0cab7d8d30579afd0e7db208e047c5a8ad1dd9b71; archived source capture; exact capture time not independently established
A StrongREJECT ¯ \overline{\hbox{{StrongREJECT}}} for Empty Jailbreaks · §3 Human evaluationSelected original resource https://arxiv.org/html/2402.10260v2; sha256:ffbcbe49cc39b51c0cd776d0cab7d8d30579afd0e7db208e047c5a8ad1dd9b71; archived source capture; exact capture time not independently established
A StrongREJECT ¯ \overline{\hbox{{StrongREJECT}}} for Empty Jailbreaks · Appendix D Robustness of Spearman correlation resultsSelected original resource https://arxiv.org/html/2402.10260v2; sha256:ffbcbe49cc39b51c0cd776d0cab7d8d30579afd0e7db208e047c5a8ad1dd9b71; archived source capture; exact capture time not independently established
A StrongREJECT ¯ \overline{\hbox{{StrongREJECT}}} for Empty Jailbreaks · §5 DiscussionSelected original resource https://arxiv.org/html/2402.10260v2; sha256:ffbcbe49cc39b51c0cd776d0cab7d8d30579afd0e7db208e047c5a8ad1dd9b71; archived source capture; exact capture time not independently established

来源核查、数据提取、编辑批准与实验复现分别记录。方法说明 →

机构与角色

研究机构加州大学伯克利分校

作者 (11)

Alexandra Souly · Qingyuan Lu · Dillon Bowen · Tu Trinh · Elvis Hsieh · Sana Pandey · Pieter Abbeel · Justin Svegliato · Scott Emmons · Olivia Watkins · Sam Toyer

评测协议

StrongREJECT →

引用与复用

引用研究发现时请引用原始出版物；引用 NUWA 的整理与解释时，单独引用本站记录及对应快照。

BibTeX ↓RIS ↓记录与来源元数据 · JSON ↓阅读覆盖 · JSON 提交更正

原始出版物 · BibTeX

@misc{strongreject,
  title = {A StrongREJECT for Empty Jailbreaks},
  url = {https://arxiv.org/abs/2402.10260v2},
  author = {Alexandra Souly and Qingyuan Lu and Dillon Bowen and Tu Trinh and Elvis Hsieh and Sana Pandey and Pieter Abbeel and Justin Svegliato and Scott Emmons and Olivia Watkins and Sam Toyer},
  year = {2024},
  date = {2024-02-15},
  version = {arXiv:2402.10260v2},
  note = {arXiv:2402.10260v2}
}

NUWA 记录 · BibTeX

@misc{strongreject-nuwa,
  title = {A StrongREJECT for Empty Jailbreaks — NUWA research record},
  url = {https://whitzard.tech/nuwa/whitzard-index/work/strongreject/},
  version = {2026-10-29.1},
  organization = {NUWA Frontier AI Safety Lab},
  howpublished = {Retained note edition: https://whitzard.tech/nuwa/whitzard-index/data/record-editions/2026-10-29.1--strongreject.json},
  note = {NUWA research note. Source-reading scope and independent review status are stated on the record. Snapshot 2026-10-29.1. Retained note edition: https://whitzard.tech/nuwa/whitzard-index/data/record-editions/2026-10-29.1--strongreject.json.}
}

证据快照 2026-10-29.1 · 索引更新 · RSS

参考文献

[1] Alexandra Souly ∗, Qingyuan Lu ∗ 等. A StrongREJECT for Empty Jailbreaks. 2024-08-27. v2 [arXiv]; Author affiliations on the original paper title page; §2 The StrongREJECT benchmark; §3 Human evaluation