风险评测 · 评测基准

Catastrophic Cyber Capabilities Benchmark (3CB): Robustly Evaluating LLM Agent Cyber Offense Capabilities

NUWA 研究解读 · 独立复核待完成

首次发表: 2024-10-10来源核验: 2026-10-25

原始来源 ↗引用本记录 ↓网络安全 →

NUWA Research Note

研究问题研究设计与假设来源发现 NUWA 解读局限与相反证据研究与决策意义机构与角色原始资源

模型辅助整理 · 选读原始全文或官方文章中的相关章节；尚未完成逐章节审计。

研究速读

NUWA 解读：输出接口可混杂表面网络能力。

研究问题

网络能力评价对激发及通信格式有多敏感？

研究设计与假设

3CB以15题、80种激发配置评价14模型，另有四题出于伦理原因保留私有。

Catastrophic Cyber Capabilities Benchmark (3CB): Robustly Evaluating LLM Agent Cyber Offense Capabilities — Figure 3 — Figure 3 · 原文 Figure 3。请结合下方的协议范围与局限阅读。原图与来源 ↗

来源发现

所报混合效应分析中GPT-4o用XML较JSON提高24.7个百分点，p<0.001。

不同模型格式敏感性有别（§3.2）。

NUWA 解读

NUWA 解读：输出接口可混杂表面网络能力。

局限与相反证据

题目完成不测灾难结果，保留任务与有限广度限制公开复现。

研究与决策意义

模型建议：解释排名或风险标签前记录格式及激发配置。

精确来源位置

Catastrophic Cyber Capabilities Benchmark (3CB): Robustly Evaluating LLM Agent Cyber Offense Capabilities · AbstractSelected original resource https://arxiv.org/html/2410.09114; sha256:b48d6428cc9f577412b3bb2c8bca9f4df649e94e9c2472006f863d8e2ed4046f; archived source capture; exact capture time not independently established
Catastrophic Cyber Capabilities Benchmark (3CB): Robustly Evaluating LLM Agent Cyber Offense Capabilities · §2 MethodologySelected original resource https://arxiv.org/html/2410.09114; sha256:b48d6428cc9f577412b3bb2c8bca9f4df649e94e9c2472006f863d8e2ed4046f; archived source capture; exact capture time not independently established
Catastrophic Cyber Capabilities Benchmark (3CB): Robustly Evaluating LLM Agent Cyber Offense Capabilities · §3.2 Elicitation resultsSelected original resource https://arxiv.org/html/2410.09114; sha256:b48d6428cc9f577412b3bb2c8bca9f4df649e94e9c2472006f863d8e2ed4046f; archived source capture; exact capture time not independently established
Catastrophic Cyber Capabilities Benchmark (3CB): Robustly Evaluating LLM Agent Cyber Offense Capabilities · §4 DiscussionSelected original resource https://arxiv.org/html/2410.09114; sha256:b48d6428cc9f577412b3bb2c8bca9f4df649e94e9c2472006f863d8e2ed4046f; archived source capture; exact capture time not independently established

来源核查、数据提取、编辑批准与实验复现分别记录。方法说明 →

机构与角色

研究机构 Apart Research

作者 (5)

Andrey Anurin · Jonathan Ng · Kibo Schaffer · Jason Schreiber · Esben Kran

评测协议

Catastrophic Cyber Capabilities Benchmark (3CB): Robustly Evaluating LLM Agent Cyber Offense Capabilities →

引用与复用

引用研究发现时请引用原始出版物；引用 NUWA 的整理与解释时，单独引用本站记录及对应快照。

BibTeX ↓RIS ↓记录与来源元数据 · JSON ↓阅读覆盖 · JSON 提交更正

原始出版物 · BibTeX

@misc{catastrophic-cyber-benchmark,
  title = {Catastrophic Cyber Capabilities Benchmark (3CB): Robustly Evaluating LLM Agent Cyber Offense Capabilities},
  url = {https://arxiv.org/abs/2410.09114},
  author = {Andrey Anurin and Jonathan Ng and Kibo Schaffer and Jason Schreiber and Esben Kran},
  year = {2024},
  date = {2024-10-10},
  version = {arXiv:2410.09114v2},
  note = {arXiv:2410.09114v2}
}

NUWA 记录 · BibTeX

@misc{catastrophic-cyber-benchmark-nuwa,
  title = {Catastrophic Cyber Capabilities Benchmark (3CB): Robustly Evaluating LLM Agent Cyber Offense Capabilities — NUWA research record},
  url = {https://whitzard.tech/nuwa/whitzard-index/work/catastrophic-cyber-benchmark/},
  version = {2026-10-29.1},
  organization = {NUWA Frontier AI Safety Lab},
  howpublished = {Retained note edition: https://whitzard.tech/nuwa/whitzard-index/data/record-editions/2026-10-29.1--catastrophic-cyber-benchmark.json},
  note = {NUWA research note. Source-reading scope and independent review status are stated on the record. Snapshot 2026-10-29.1. Retained note edition: https://whitzard.tech/nuwa/whitzard-index/data/record-editions/2026-10-29.1--catastrophic-cyber-benchmark.json.}
}

证据快照 2026-10-29.1 · 索引更新 · RSS

参考文献

[1] Your Name. Catastrophic Cyber Capabilities Benchmark (3CB): Robustly Evaluating LLM Agent Cyber Offense Capabilities. 2024-10-10. v2 [arXiv]; Abstract; §2 Methodology; §3.2 Elicitation results
[2] ‘3cb’: The Catastrophic Cyber Capabilities Benchmark. 2024-10-30 ; Original institutional announcement: Apart Research paper