Risk Evaluation · Benchmark

CVE-Bench: A Benchmark for AI Agents' Ability to Exploit Real-World Web Application Vulnerabilities

NUWA research note · independent review pending

First published: 2025-03-21Source checked: 2026-10-25

Original source ↗Cite this record ↓Cybersecurity →

NUWA Research Note

Research question Design and assumptions Source findings NUWA interpretation Limits and contrary evidence Research and decision relevance Institutions and roles Original resources

Model-assisted reading · Selected sections of the original full text or official article; a complete section-by-section audit remains pending.

At a glance

NUWA interpretation: realistic application state adds coverage beyond abstract CTFs, but success remains task- and condition-specific.

Research question

How well do agents exploit severe, real web-application CVEs in contained environments?

Design and assumptions

Forty selected CVEs use a sandbox and eight predefined outcome categories, with distinct zero-day and one-day information conditions.

CVE-Bench — Figure 1 — Figure 1 · Original figure 1. Read alongside the protocol scope and limitations below. Original figure and source ↗

Source findings

The paper reports up to 13% vulnerability success for its tested agent frameworks.

Successful-attack composition differs across agents and information conditions (§4.2).

NUWA interpretation

NUWA interpretation: realistic application state adds coverage beyond abstract CTFs, but success remains task- and condition-specific.

Limits and contrary evidence

Only forty web CVEs in a bounded date range are covered; attacks outside the eight outcome classes can be false negatives (§5).

Research and decision relevance

Model recommendation: retain information setting and outcome coverage before comparing with reproduction-only benchmarks.

Precise source locations

CVE-Bench: A Benchmark for AI Agents’ Ability to Exploit Real-World Web Application Vulnerabilities · AbstractSelected original resource https://arxiv.org/html/2503.17332; sha256:653883d8aa53bf652f06e5c89d5906f1b0078e2f4da5b7d455baf562250b505f; archived source capture; exact capture time not independently established
CVE-Bench: A Benchmark for AI Agents’ Ability to Exploit Real-World Web Application Vulnerabilities · §3.2 Task SpecificationSelected original resource https://arxiv.org/html/2503.17332; sha256:653883d8aa53bf652f06e5c89d5906f1b0078e2f4da5b7d455baf562250b505f; archived source capture; exact capture time not independently established
CVE-Bench: A Benchmark for AI Agents’ Ability to Exploit Real-World Web Application Vulnerabilities · §3.3 Benchmark ConstructionSelected original resource https://arxiv.org/html/2503.17332; sha256:653883d8aa53bf652f06e5c89d5906f1b0078e2f4da5b7d455baf562250b505f; archived source capture; exact capture time not independently established
CVE-Bench: A Benchmark for AI Agents’ Ability to Exploit Real-World Web Application Vulnerabilities · §4.2 ResultsSelected original resource https://arxiv.org/html/2503.17332; sha256:653883d8aa53bf652f06e5c89d5906f1b0078e2f4da5b7d455baf562250b505f; archived source capture; exact capture time not independently established
CVE-Bench: A Benchmark for AI Agents’ Ability to Exploit Real-World Web Application Vulnerabilities · §5 DiscussionSelected original resource https://arxiv.org/html/2503.17332; sha256:653883d8aa53bf652f06e5c89d5906f1b0078e2f4da5b7d455baf562250b505f; archived source capture; exact capture time not independently established

Source checking, extraction, editorial approval and replication are recorded separately. Methodology →

Institutions and roles

Research by University of Illinois Urbana-Champaign

Authors (16)

Yuxuan Zhu · Antony Kellermann · Dylan Bowman · Philip Li · Akul Gupta · Adarsh Danda · Richard Fang · Conner Jensen · Eric Ihli · Jason Benn · Jet Geronimo · Avi Dhir · Sudhit Rao · Kaicheng Yu · Twm Stone · Daniel Kang

Evaluation protocols

CVE-Bench →

Cite and reuse

Cite the original publication for its findings. Cite this NUWA record separately for its explanation and retained snapshot.

BibTeX ↓RIS ↓Record + source metadata · JSON ↓Reading coverage · JSON Suggest a correction

Original publication · BibTeX

@misc{cve-bench,
  title = {CVE-Bench: A Benchmark for AI Agents' Ability to Exploit Real-World Web Application Vulnerabilities},
  url = {https://arxiv.org/abs/2503.17332},
  author = {Yuxuan Zhu and Antony Kellermann and Dylan Bowman and Philip Li and Akul Gupta and Adarsh Danda and Richard Fang and Conner Jensen and Eric Ihli and Jason Benn and Jet Geronimo and Avi Dhir and Sudhit Rao and Kaicheng Yu and Twm Stone and Daniel Kang},
  year = {2025},
  date = {2025-03-21},
  version = {arXiv:2503.17332v4},
  note = {arXiv:2503.17332v4}
}

NUWA record · BibTeX

@misc{cve-bench-nuwa,
  title = {CVE-Bench: A Benchmark for AI Agents' Ability to Exploit Real-World Web Application Vulnerabilities — NUWA research record},
  url = {https://whitzard.tech/en/nuwa/whitzard-index/work/cve-bench/},
  version = {2026-10-29.1},
  organization = {NUWA Frontier AI Safety Lab},
  howpublished = {Retained note edition: https://whitzard.tech/nuwa/whitzard-index/data/record-editions/2026-10-29.1--cve-bench.json},
  note = {NUWA research note. Source-reading scope and independent review status are stated on the record. Snapshot 2026-10-29.1. Retained note edition: https://whitzard.tech/nuwa/whitzard-index/data/record-editions/2026-10-29.1--cve-bench.json.}
}

Evidence snapshot 2026-10-29.1 · Index updates · RSS

References

[1] Yuxuan Zhu, Antony Kellermann, et al.. CVE-Bench: A Benchmark for AI Agents' Ability to Exploit Real-World Web Application Vulnerabilities. 2025-03-21. v4 [arXiv]; Author affiliations on the original paper title page; Abstract; §3.2 Task Specification