{"schemaVersion":"nuwa-current-record-1","version":"2026-10-29.1","publication":"published","work":{"id":"cve-bench","title":"CVE-Bench: A Benchmark for AI Agents' Ability to Exploit Real-World Web Application Vulnerabilities","aliases":["CVE-Bench"],"authors":["Yuxuan Zhu","Antony Kellermann","Dylan Bowman","Philip Li","Akul Gupta","Adarsh Danda","Richard Fang","Conner Jensen","Eric Ihli","Jason Benn","Jet Geronimo","Avi Dhir","Sudhit Rao","Kaicheng Yu","Twm Stone","Daniel Kang"],"primaryFunction":"evaluation","secondaryFunctions":[],"kind":"benchmark","sourceId":"cve-bench-publication","attributionSourceId":"cve-bench-publication-author-source","originalUrl":"https://arxiv.org/abs/2503.17332","publicationDate":"2025-03-21","updatedOn":"2026-10-25","areas":["cybersecurity"],"scopeNote":{"en":"The main dimension follows the task outcome in the original protocol. Topic relevance is documented in the parent research record; no unreviewed question-level links are inferred.","zh":"主维度依据原协议的任务结果确定。主题关联见研究档案，不推断尚未审阅的问题级关联。"},"protocolIds":["cve-bench"],"contributions":[{"organizationId":"uiuc","role":"research","sourceId":"cve-bench-publication-author-source","locator":"Author affiliations on the original paper title page"}],"summary":{"en":"Measures agents’ exploitation of real web-application vulnerabilities in runnable environments with explicit success checks.","zh":"在可运行环境中，以明确成功判据评测智能体利用真实 Web 应用漏洞的能力。"},"note":{"question":null,"method":null,"finding":null,"supports":null,"limits":null,"connections":null},"resources":[],"noteStatus":{"question":"limited","method":"limited","finding":"limited","supports":"limited","limits":"limited","connections":"limited"},"noteAvailability":{"en":"A substantive NUWA research note is not yet available. The repeated draft sections were removed pending review of the original methods, findings and limitations; use the original publication for the research account.","zh":"实质性的 NUWA 研究笔记尚未完成。重复草稿章节已移除，待核查原始方法、发现与限制；研究内容请以原始出版物为准。"},"noteSourceLocator":"No complete section-level method/results/limitations review completed for this note; cached original page and any chart audit retained as discovery evidence.","tags":[]},"reading":{"id":"cve-bench","title":"CVE-Bench: A Benchmark for AI Agents' Ability to Exploit Real-World Web Application Vulnerabilities","en":{"question":"How well do agents exploit severe, real web-application CVEs in contained environments?","design":"Forty selected CVEs use a sandbox and eight predefined outcome categories, with distinct zero-day and one-day information conditions.","findings":["The paper reports up to 13% vulnerability success for its tested agent frameworks.","Successful-attack composition differs across agents and information conditions (§4.2)."],"interpretation":"NUWA interpretation: realistic application state adds coverage beyond abstract CTFs, but success remains task- and condition-specific.","limits":"Only forty web CVEs in a bounded date range are covered; attacks outside the eight outcome classes can be false negatives (§5).","decisionRelevance":"Model recommendation: retain information setting and outcome coverage before comparing with reproduction-only benchmarks.","takeaway":"NUWA interpretation: realistic application state adds coverage beyond abstract CTFs, but success remains task- and condition-specific.","connections":[],"scoring":"Success in eight predefined outcome classes; zero-/one-day conditions differ.","conditions":"Forty severe web CVEs in isolated reconstructed applications.","statistics":"The paper reports up to 13% vulnerability success for its tested agent frameworks. Successful-attack composition differs across agents and information conditions (§4.2).","usage":"Model recommendation: retain information setting and outcome coverage before comparing with reproduction-only benchmarks."},"zh":{"question":"智能体能否在隔离环境利用真实严重Web CVE？","design":"40个CVE使用沙箱与八类预定义结果，区分无披露描述和有披露描述的信息条件。","findings":["受测智能体框架最高成功13%。","成功攻击的组成随框架与信息条件变化（§4.2）。"],"interpretation":"NUWA 解读：真实应用状态补充了抽象CTF，但成功仍依赖任务与条件。","limits":"仅覆盖特定时期40个Web CVE，八类外的攻击可能被漏判（§5）。","decisionRelevance":"模型建议：比较复现型基准前保留信息条件及结果覆盖范围。","takeaway":"NUWA 解读：真实应用状态补充了抽象CTF，但成功仍依赖任务与条件。","connections":[],"scoring":"八类预定结果成功，区分无披露与有披露信息。","conditions":"隔离重建应用中40个严重Web CVE。","statistics":"受测智能体框架最高成功13%。 成功攻击的组成随框架与信息条件变化（§4.2）。","usage":"模型建议：比较复现型基准前保留信息条件及结果覆盖范围。"},"assertionPolicy":{"question":"source-research-question","design":"source-finding","findings":"source-finding","interpretation":"nuwa-interpretation","limits":"source-limit-or-explicit-editorial-scope-limit","connections":"source-supported-relationship-only","decisionRelevance":"model-recommendation-not-source-conclusion"},"sourceRefs":[{"sourceId":"cve-bench-publication-author-source","url":"https://arxiv.org/html/2503.17332","locator":"Abstract","version":"Selected original resource https://arxiv.org/html/2503.17332; sha256:653883d8aa53bf652f06e5c89d5906f1b0078e2f4da5b7d455baf562250b505f; archived source capture; exact capture time not independently established","artifactSha256":"653883d8aa53bf652f06e5c89d5906f1b0078e2f4da5b7d455baf562250b505f"},{"sourceId":"cve-bench-publication-author-source","url":"https://arxiv.org/html/2503.17332","locator":"§3.2 Task Specification","version":"Selected original resource https://arxiv.org/html/2503.17332; sha256:653883d8aa53bf652f06e5c89d5906f1b0078e2f4da5b7d455baf562250b505f; archived source capture; exact capture time not independently established","artifactSha256":"653883d8aa53bf652f06e5c89d5906f1b0078e2f4da5b7d455baf562250b505f"},{"sourceId":"cve-bench-publication-author-source","url":"https://arxiv.org/html/2503.17332","locator":"§3.3 Benchmark Construction","version":"Selected original resource https://arxiv.org/html/2503.17332; sha256:653883d8aa53bf652f06e5c89d5906f1b0078e2f4da5b7d455baf562250b505f; archived source capture; exact capture time not independently established","artifactSha256":"653883d8aa53bf652f06e5c89d5906f1b0078e2f4da5b7d455baf562250b505f"},{"sourceId":"cve-bench-publication-author-source","url":"https://arxiv.org/html/2503.17332","locator":"§4.2 Results","version":"Selected original resource https://arxiv.org/html/2503.17332; sha256:653883d8aa53bf652f06e5c89d5906f1b0078e2f4da5b7d455baf562250b505f; archived source capture; exact capture time not independently established","artifactSha256":"653883d8aa53bf652f06e5c89d5906f1b0078e2f4da5b7d455baf562250b505f"},{"sourceId":"cve-bench-publication-author-source","url":"https://arxiv.org/html/2503.17332","locator":"§5 Discussion","version":"Selected original resource https://arxiv.org/html/2503.17332; sha256:653883d8aa53bf652f06e5c89d5906f1b0078e2f4da5b7d455baf562250b505f; archived source capture; exact capture time not independently established","artifactSha256":"653883d8aa53bf652f06e5c89d5906f1b0078e2f4da5b7d455baf562250b505f"}],"publication":{"state":"published","publishedOn":"2026-10-29","authorizationReference":"user-authorized-global-index-publication-2026-10-29","reviewedCandidateId":"local-6efe31bc4a0c0f37"},"review":{"state":"published-note","sourceScope":"full-text","availableScope":null,"readingCoverage":"selected-source-sections; not a complete section-by-section paper audit","completePaperAudit":false,"independentEvidenceReview":"pending-human-review","bilingualEditorialReview":"pending-human-review","authorship":"model-assisted-research-note","preparedFingerprint":"a10106f5b4b4a22d3b6e7a78f16d64c5b3455eaab187dcb9aae0afa2436e77e5"}},"sources":[{"id":"cve-bench-publication","title":"CVE-Bench: A Benchmark for AI Agents' Ability to Exploit Real-World Web Application Vulnerabilities","url":"https://arxiv.org/abs/2503.17332","version":"arXiv:2503.17332v4","publicationDate":"2025-03-21","datePrecision":"day","type":"paper","verifiedOn":"2026-10-25","documentId":"arxiv:2503.17332","authors":["Yuxuan Zhu","Antony Kellermann","Dylan Bowman","Philip Li","Akul Gupta","Adarsh Danda","Richard Fang","Conner Jensen","Eric Ihli","Jason Benn","Jet Geronimo","Avi Dhir","Sudhit Rao","Kaicheng Yu","Twm Stone","Daniel Kang"],"aliases":["https://doi.org/10.48550/arXiv.2503.17332","https://arxiv.org/abs/2503.17332","https://arxiv.org/pdf/2503.17332"]}]}