Measuring AI Agents’ Progress on Multi-Step Cyber Attack Scenarios

档案区分来源方法阅读与既有目录信息；已经公开不代表完成独立证据复核或双语编辑复核。

测量构念

模型代际与推理预算如何影响连续靶场任务进展？

任务与样本

七个模型在32步、7步两个定制靶场测试；近期模型的100M令牌端到端条件各运行五次。

判分与失效情形

按完成靶场步骤数计，最佳运行不同于平均进展。

系统条件

区分10M与100M令牌，近期高预算端到端单元五次尝试。

统计含义

来源报告的结果表述；这不等同于已确定运行次数、分母或估计方法： 10M令牌下，企业靶场平均进展从GPT-4o的1.7步升至Opus 4.6的9.8步。 最佳单次完成22/32步，并非平均完成率。

效度与解释

仅两个定制靶场，高预算样本较小；披露还可能削弱未来保留集有效性（§5）。

适用研究问题

模型建议：匹配令牌预算，并在最佳进展旁报告运行差异。

• Measuring AI Agents’ Progress on Multi-Step Cyber Attack Scenarios · AbstractSelected original resource https://arxiv.org/html/2603.11214; sha256:fe735767bb4d07e484ce73b74b4095cb5dda946262a39b0fd70c3ed2f05015cf; archived source capture; exact capture time not independently established
• Measuring AI Agents’ Progress on Multi-Step Cyber Attack Scenarios · §3.3 Experimental designSelected original resource https://arxiv.org/html/2603.11214; sha256:fe735767bb4d07e484ce73b74b4095cb5dda946262a39b0fd70c3ed2f05015cf; archived source capture; exact capture time not independently established
• Measuring AI Agents’ Progress on Multi-Step Cyber Attack Scenarios · §2 ResultsSelected original resource https://arxiv.org/html/2603.11214; sha256:fe735767bb4d07e484ce73b74b4095cb5dda946262a39b0fd70c3ed2f05015cf; archived source capture; exact capture time not independently established
• Measuring AI Agents’ Progress on Multi-Step Cyber Attack Scenarios · §5 LimitationsSelected original resource https://arxiv.org/html/2603.11214; sha256:fe735767bb4d07e484ce73b74b4095cb5dda946262a39b0fd70c3ed2f05015cf; archived source capture; exact capture time not independently established