风险评测 · 评测基准

SEC-bench Pro: Can Language Models Solve Long-Horizon Software Security Tasks?

NUWA 研究解读 · 独立复核待完成

首次发表: 2026-05-26来源核验: 2026-10-25

原始来源 ↗引用本记录 ↓网络安全 →

NUWA Research Note

研究问题研究设计与假设来源发现 NUWA 解读局限与相反证据研究与决策意义机构与角色原始资源

模型辅助整理 · 选读原始全文或官方文章中的相关章节；尚未完成逐章节审计。

研究速读

NUWA 解读：特定目标验证及超时记账实质定义了测量能力。

研究问题

智能体能否在大型安全关键代码库复现指定披露漏洞？

研究设计与假设

344例覆盖V8、SpiderMonkey与Linux；判分检查漏洞、修复及最新版本，并区分全体与完成运行分母。

SEC-bench Pro — Source figure — Source figure · 原文 Source figure。请结合下方的协议范围与局限阅读。原图与来源 ↗

来源发现

所报最佳模型框架整体成功58%。

成功须通过三镜像目标归因，崩溃本身不够（§IV-A）。

NUWA 解读

NUWA 解读：特定目标验证及超时记账实质定义了测量能力。

局限与相反证据

报告指引的复现不是无指引发现；目标族与框架限制外推。

研究与决策意义

模型建议：解释复现差异前核对全体分母及修复、最新版本证据。

精确来源位置

SEC-bench Pro: Can Language Models Solve Long-Horizon Software Security Tasks? Thanks: ∗ Correspondence to: {hwiwonl2, lingming}@illinois.edu . · AbstractSelected original resource https://arxiv.org/html/2605.26548v2; sha256:fb70724c35773ab7a7ee2c65b4396aa1004f768690b3332f4d8203b7d0929bf9; archived source capture; exact capture time not independently established
SEC-bench Pro: Can Language Models Solve Long-Horizon Software Security Tasks? Thanks: ∗ Correspondence to: {hwiwonl2, lingming}@illinois.edu . · §III DesignSelected original resource https://arxiv.org/html/2605.26548v2; sha256:fb70724c35773ab7a7ee2c65b4396aa1004f768690b3332f4d8203b7d0929bf9; archived source capture; exact capture time not independently established
SEC-bench Pro: Can Language Models Solve Long-Horizon Software Security Tasks? Thanks: ∗ Correspondence to: {hwiwonl2, lingming}@illinois.edu . · §IV-A Evaluation SetupSelected original resource https://arxiv.org/html/2605.26548v2; sha256:fb70724c35773ab7a7ee2c65b4396aa1004f768690b3332f4d8203b7d0929bf9; archived source capture; exact capture time not independently established
SEC-bench Pro: Can Language Models Solve Long-Horizon Software Security Tasks? Thanks: ∗ Correspondence to: {hwiwonl2, lingming}@illinois.edu . · §IV-F Threats to ValiditySelected original resource https://arxiv.org/html/2605.26548v2; sha256:fb70724c35773ab7a7ee2c65b4396aa1004f768690b3332f4d8203b7d0929bf9; archived source capture; exact capture time not independently established

来源核查、数据提取、编辑批准与实验复现分别记录。方法说明 →

机构与角色

研究机构伊利诺伊大学厄巴纳—香槟分校

作者 (7)

Hwiwon Lee · Jiawei Liu · Dongjun Kim · Wubing Xia · Ziqi Zhang · Chunqiu Steven Xia · Lingming Zhang

评测协议

SEC-bench Pro →

引用与复用

引用研究发现时请引用原始出版物；引用 NUWA 的整理与解释时，单独引用本站记录及对应快照。

BibTeX ↓RIS ↓记录与来源元数据 · JSON ↓阅读覆盖 · JSON 提交更正

原始出版物 · BibTeX

@misc{secbench-pro,
  title = {SEC-bench Pro: Can Language Models Solve Long-Horizon Software Security Tasks?},
  url = {https://arxiv.org/abs/2605.26548v2},
  author = {Hwiwon Lee and Jiawei Liu and Dongjun Kim and Wubing Xia and Ziqi Zhang and Chunqiu Steven Xia and Lingming Zhang},
  year = {2026},
  date = {2026-05-26},
  version = {arXiv:2605.26548v2},
  note = {arXiv:2605.26548v2}
}

NUWA 记录 · BibTeX

@misc{secbench-pro-nuwa,
  title = {SEC-bench Pro: Can Language Models Solve Long-Horizon Software Security Tasks? — NUWA research record},
  url = {https://whitzard.tech/nuwa/whitzard-index/work/secbench-pro/},
  version = {2026-10-29.1},
  organization = {NUWA Frontier AI Safety Lab},
  howpublished = {Retained note edition: https://whitzard.tech/nuwa/whitzard-index/data/record-editions/2026-10-29.1--secbench-pro.json},
  note = {NUWA research note. Source-reading scope and independent review status are stated on the record. Snapshot 2026-10-29.1. Retained note edition: https://whitzard.tech/nuwa/whitzard-index/data/record-editions/2026-10-29.1--secbench-pro.json.}
}

证据快照 2026-10-29.1 · 索引更新 · RSS

参考文献

[1] Hwiwon Lee ∗ Jiawei Liu Dongjun Kim Wubing Xia Ziqi Zhang Chunqiu Steven Xia Lingming Zhang ∗. SEC-bench Pro: Can Language Models Solve Long-Horizon Software Security Tasks?. 2026-07-20. v2 [arXiv]; Author affiliations on the original paper title page; Abstract; §III Design