风险评测 · 评测基准

The Next Challenge for Agentic Cybersecurity: A Realistic, Contamination-Free Reverse Engineering Benchmark

NUWA 研究解读 · 独立复核待完成

首次发表: 2026-08-11来源核验: 2026-10-25

原始来源 ↗引用本记录 ↓网络安全 →

NUWA Research Note

研究问题研究设计与假设来源发现 NUWA 解读局限与相反证据研究与决策意义机构与角色原始资源

模型辅助整理 · 选读原始全文或官方文章中的相关章节；尚未完成逐章节审计。

研究速读

NUWA 解读：源码安全表现不能替代二进制分析测量。

研究问题

无源码时，智能体能否恢复真实规模二进制的语义？

研究设计与假设

19个私有自编程序形成262个二进制实例、1,572项确定性任务，44种保护机制改变难度。

SRE-Bench — Figure 1 — Figure 1 · 原文 Figure 1。请结合下方的协议范围与局限阅读。原图与来源 ↗

来源发现

GPT-5.6 Sol完整解决80/254例，Claude Fable 5.1为60/223。

分母不同，且研究区分公开与内部无约束设置（§4）。

NUWA 解读

NUWA 解读：源码安全表现不能替代二进制分析测量。

局限与相反证据

19个程序限制广度；私有编写缓解污染，但不证明题目验证穷尽（§5）。

研究与决策意义

模型建议：保留每模型有效分母、保护变体和防护条件。

精确来源位置

The Next Challenge for Agentic Cybersecurity: A Realistic, Contamination-Free Reverse Engineering Benchmark · AbstractSelected original resource https://arxiv.org/html/2608.11469; sha256:1d4839f5e2ce6a1193b841dbf689059f6a68140a440495064b4e40727076e5fa; archived source capture; exact capture time not independently established
The Next Challenge for Agentic Cybersecurity: A Realistic, Contamination-Free Reverse Engineering Benchmark · §3 BenchmarkSelected original resource https://arxiv.org/html/2608.11469; sha256:1d4839f5e2ce6a1193b841dbf689059f6a68140a440495064b4e40727076e5fa; archived source capture; exact capture time not independently established
The Next Challenge for Agentic Cybersecurity: A Realistic, Contamination-Free Reverse Engineering Benchmark · §4 EvaluationSelected original resource https://arxiv.org/html/2608.11469; sha256:1d4839f5e2ce6a1193b841dbf689059f6a68140a440495064b4e40727076e5fa; archived source capture; exact capture time not independently established
The Next Challenge for Agentic Cybersecurity: A Realistic, Contamination-Free Reverse Engineering Benchmark · §5 Discussion and ConclusionSelected original resource https://arxiv.org/html/2608.11469; sha256:1d4839f5e2ce6a1193b841dbf689059f6a68140a440495064b4e40727076e5fa; archived source capture; exact capture time not independently established

来源核查、数据提取、编辑批准与实验复现分别记录。方法说明 →

机构与角色

研究机构哥伦比亚大学 Google DeepMind 加州大学伯克利分校 Vals AI OpenAI Anthropic 塔夫茨大学加州大学洛杉矶分校

作者 (12)

Jeremy Spence · Nicholas Assaderaghi · Feng Xiao · Jinhao Zhu · Nikil Ravi · Xiangyu Qi · Matthew Jagielski · Raluca Ada Popa · Eric Wallace · Guannan Wei · Yangruibo Ding · Zhuo Zhang

评测协议

SRE-Bench →

引用与复用

引用研究发现时请引用原始出版物；引用 NUWA 的整理与解释时，单独引用本站记录及对应快照。

BibTeX ↓RIS ↓记录与来源元数据 · JSON ↓阅读覆盖 · JSON 提交更正

原始出版物 · BibTeX

@misc{sre-bench,
  title = {The Next Challenge for Agentic Cybersecurity: A Realistic, Contamination-Free Reverse Engineering Benchmark},
  url = {https://arxiv.org/abs/2608.11469},
  author = {Jeremy Spence and Nicholas Assaderaghi and Feng Xiao and Jinhao Zhu and Nikil Ravi and Xiangyu Qi and Matthew Jagielski and Raluca Ada Popa and Eric Wallace and Guannan Wei and Yangruibo Ding and Zhuo Zhang},
  year = {2026},
  date = {2026-08-11},
  version = {arXiv:2608.11469v2},
  note = {arXiv:2608.11469v2}
}

NUWA 记录 · BibTeX

@misc{sre-bench-nuwa,
  title = {The Next Challenge for Agentic Cybersecurity: A Realistic, Contamination-Free Reverse Engineering Benchmark — NUWA research record},
  url = {https://whitzard.tech/nuwa/whitzard-index/work/sre-bench/},
  version = {2026-10-29.1},
  organization = {NUWA Frontier AI Safety Lab},
  howpublished = {Retained note edition: https://whitzard.tech/nuwa/whitzard-index/data/record-editions/2026-10-29.1--sre-bench.json},
  note = {NUWA research note. Source-reading scope and independent review status are stated on the record. Snapshot 2026-10-29.1. Retained note edition: https://whitzard.tech/nuwa/whitzard-index/data/record-editions/2026-10-29.1--sre-bench.json.}
}

证据快照 2026-10-29.1 · 索引更新 · RSS

参考文献

[1] Jeremy Spence, Nicholas Assaderaghi 等. The Next Challenge for Agentic Cybersecurity: A Realistic, Contamination-Free Reverse Engineering Benchmark. 2026-08-11. v2 [arXiv]; Author affiliations on the original paper title page; Abstract; §3 Benchmark