风险评测 · 评测基准

HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal

NUWA 研究解读 · 独立复核待完成

首次发表: 2024-02-06来源核验: 2026-10-25

原始来源 ↗引用本记录 ↓

NUWA Research Note

研究问题研究设计与假设来源发现 NUWA 解读局限与相反证据研究与决策意义机构与角色原始资源

模型辅助整理 · 选读原始全文或官方文章中的相关章节；尚未完成逐章节审计。

研究速读

NUWA 解读：攻击成功率同时取决于行为组成和成功分类器。

研究问题

自动红队方法与防护能否采用统一的行为级判分比较？

研究设计与假设

框架标准化有害行为、生成及分类；所报研究比较18种攻击与33个目标模型或防护。

HarmBench — Figure 1 — Figure 1 · 原文 Figure 1。请结合下方的协议范围与局限阅读。原图与来源 ↗

来源发现

上下文类行为的攻击成功率较高。

版权类采用比非版权分类器更严格的内容匹配判据（§6.1）。

NUWA 解读

NUWA 解读：攻击成功率同时取决于行为组成和成功分类器。

局限与相反证据

功能类别判据不同，汇总成功率不能视为等价的真实伤害尺度。

研究与决策意义

模型建议：比较攻击或防护时保留分类器及类别明细。

精确来源位置

HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal · AbstractSelected original resource https://arxiv.org/html/2402.04249v2; sha256:c0222f5cb89e2f882de4cbb59e4d61897049528425e5e0cce1928bfb0655e37d; archived source capture; exact capture time not independently established
HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal · §4 HarmBenchSelected original resource https://arxiv.org/html/2402.04249v2; sha256:c0222f5cb89e2f882de4cbb59e4d61897049528425e5e0cce1928bfb0655e37d; archived source capture; exact capture time not independently established
HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal · §6.1 Main ResultsSelected original resource https://arxiv.org/html/2402.04249v2; sha256:c0222f5cb89e2f882de4cbb59e4d61897049528425e5e0cce1928bfb0655e37d; archived source capture; exact capture time not independently established
HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal · §7 ConclusionSelected original resource https://arxiv.org/html/2402.04249v2; sha256:c0222f5cb89e2f882de4cbb59e4d61897049528425e5e0cce1928bfb0655e37d; archived source capture; exact capture time not independently established

来源核查、数据提取、编辑批准与实验复现分别记录。方法说明 →

机构与角色

研究机构伊利诺伊大学厄巴纳—香槟分校 Center for AI Safety 卡内基梅隆大学加州大学伯克利分校微软

作者 (12)

Mantas Mazeika · Long Phan · Xuwang Yin · Andy Zou · Zifan Wang · Norman Mu · Elham Sakhaee · Nathaniel Li · Steven Basart · Bo Li · David Forsyth · Dan Hendrycks

评测协议

HarmBench →

引用与复用

引用研究发现时请引用原始出版物；引用 NUWA 的整理与解释时，单独引用本站记录及对应快照。

BibTeX ↓RIS ↓记录与来源元数据 · JSON ↓阅读覆盖 · JSON 提交更正

原始出版物 · BibTeX

@misc{harmbench,
  title = {HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal},
  url = {https://arxiv.org/abs/2402.04249v2},
  author = {Mantas Mazeika and Long Phan and Xuwang Yin and Andy Zou and Zifan Wang and Norman Mu and Elham Sakhaee and Nathaniel Li and Steven Basart and Bo Li and David Forsyth and Dan Hendrycks},
  year = {2024},
  date = {2024-02-06},
  version = {arXiv:2402.04249v2},
  note = {arXiv:2402.04249v2}
}

NUWA 记录 · BibTeX

@misc{harmbench-nuwa,
  title = {HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal — NUWA research record},
  url = {https://whitzard.tech/nuwa/whitzard-index/work/harmbench/},
  version = {2026-10-29.1},
  organization = {NUWA Frontier AI Safety Lab},
  howpublished = {Retained note edition: https://whitzard.tech/nuwa/whitzard-index/data/record-editions/2026-10-29.1--harmbench.json},
  note = {NUWA research note. Source-reading scope and independent review status are stated on the record. Snapshot 2026-10-29.1. Retained note edition: https://whitzard.tech/nuwa/whitzard-index/data/record-editions/2026-10-29.1--harmbench.json.}
}

证据快照 2026-10-29.1 · 索引更新 · RSS

参考文献

[1] Mantas Mazeika, Long Phan 等. HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal. 2024-02-27. v2 [arXiv]; Author affiliations on the original paper title page; Abstract; §4 HarmBench