WhitzardNUWA Lab ↗
NUWANUWA INDEX

Search NUWA Index

Research · Benchmarks · Findings · Perspectives

中文
← Library

Risk Evaluation · Benchmark

ExploitBench: A Capability Ladder Benchmark for LLM Cybersecurity Agents

NUWA research note · independent review pending

First published: Source checked: 2026-10-25
Original source ↗Cite this record ↓Cybersecurity →
NUWA Research NoteResearch questionDesign and assumptionsSource findingsNUWA interpretationLimits and contrary evidenceResearch and decision relevanceInstitutions and rolesOriginal resources

Model-assisted reading · Selected sections of the original full text or official article; a complete section-by-section audit remains pending.

At a glance

NUWA interpretation: graded intermediate capability reveals information hidden by binary crash success.

Research question

Where do agents stop between triggering a known bug and achieving demonstrated security impact?

Design and assumptions

Forty-one V8 bugs are graded with sixteen deterministic capability flags, differential checks and per-run randomized verification.

ExploitBench — Figure 1
Figure 1 · Original figure 1. Read alongside the protocol scope and limitations below. Original figure and source ↗

Source findings

The public-model panel rarely crossed the reported Tier-3-to-Tier-2 boundary; GPT-5.5 did so on one bug.

Mythos Preview crossed more consistently, distinguishing a benchmark possibility from public-model behavior (Conclusion).

NUWA interpretation

NUWA interpretation: graded intermediate capability reveals information hidden by binary crash success.

Limits and contrary evidence

One-bug crossings are existence proofs; this V8 population and bounded scaffold do not estimate deployed attack rates.

Research and decision relevance

Model recommendation: report achieved capability tier and verification conditions rather than labeling every crash an exploit.

Precise source locations

Source checking, extraction, editorial approval and replication are recorded separately. Methodology →

Institutions and roles

Research by Carnegie Mellon UniversityBugcrowd

Authors (2)

Seunghyun Lee · David Brumley

Evaluation protocols

ExploitBench →

Cite and reuse

Cite the original publication for its findings. Cite this NUWA record separately for its explanation and retained snapshot.

BibTeX ↓RIS ↓Record + source metadata · JSON ↓Reading coverage · JSONSuggest a correction
Original publication · BibTeX
@misc{exploitbench,
  title = {ExploitBench: A Capability Ladder Benchmark for LLM Cybersecurity Agents},
  url = {https://arxiv.org/abs/2605.14153},
  author = {Seunghyun Lee and David Brumley},
  year = {2026},
  date = {2026-05-13},
  version = {arXiv:2605.14153v1},
  note = {arXiv:2605.14153v1}
}
NUWA record · BibTeX
@misc{exploitbench-nuwa,
  title = {ExploitBench: A Capability Ladder Benchmark for LLM Cybersecurity Agents — NUWA research record},
  url = {https://whitzard.tech/en/nuwa/whitzard-index/work/exploitbench/},
  version = {2026-10-29.1},
  organization = {NUWA Frontier AI Safety Lab},
  howpublished = {Retained note edition: https://whitzard.tech/nuwa/whitzard-index/data/record-editions/2026-10-29.1--exploitbench.json},
  note = {NUWA research note. Source-reading scope and independent review status are stated on the record. Snapshot 2026-10-29.1. Retained note edition: https://whitzard.tech/nuwa/whitzard-index/data/record-editions/2026-10-29.1--exploitbench.json.}
}

Evidence snapshot 2026-10-29.1 · Index updates · RSS

References

  1. [1] Seunghyun Lee, David Brumley. ExploitBench: A Capability Ladder Benchmark for LLM Cybersecurity Agents. 2026-05-13. v1 [arXiv]; Author affiliations on the original paper title page; Abstract; §2 Design