NUWA Evaluation dossier
ExploitBench
Separates exploit development into capability levels, distinguishing bug triggering from reusable exploitation primitives and control.
Task and measurement
Benchmark protocol profile
- Task / subject
- Model or agent system under the original task configuration
- Measurement target
- Separates exploit development into capability levels, distinguishing bug triggering from reusable exploitation primitives and control.
- Interpretation boundary
- Known-vulnerability tasks and target settings do not establish discovery of unknown vulnerabilities in deployed systems.
Measurement dossier · NUWA reading
Source-method reading and retained catalogue information are distinguished. Publication does not establish independent evidence or bilingual editorial review.
- Measurement construct
- Where do agents stop between triggering a known bug and achieving demonstrated security impact?
- Task population
- Forty-one V8 bugs are graded with sixteen deterministic capability flags, differential checks and per-run randomized verification.
- Scoring and failure modes
- Sixteen deterministically verified capability flags on forty-one V8 bugs.
- System conditions
- Binary crash is not final impact; per-run randomized verification and differential checks.
- Statistical meaning
- Source-reported result statements; this does not establish trial counts, denominators or an estimator: The public-model panel rarely crossed the reported Tier-3-to-Tier-2 boundary; GPT-5.5 did so on one bug.
- Validity and interpretation
- One-bug crossings are existence proofs; this V8 population and bounded scaffold do not estimate deployed attack rates.
- Appropriate research use
- Model recommendation: report achieved capability tier and verification conditions rather than labeling every crash an exploit.
- ExploitBench : A Capability Ladder Benchmark for LLM Cybersecurity Agents · AbstractSelected original resource https://arxiv.org/html/2605.14153; sha256:83a1a60bec38561f7d789c0d6161b3934f5df91fbc9c8c6926ef2def526f44c7; archived source capture; exact capture time not independently established
- ExploitBench : A Capability Ladder Benchmark for LLM Cybersecurity Agents · §2 DesignSelected original resource https://arxiv.org/html/2605.14153; sha256:83a1a60bec38561f7d789c0d6161b3934f5df91fbc9c8c6926ef2def526f44c7; archived source capture; exact capture time not independently established
- ExploitBench : A Capability Ladder Benchmark for LLM Cybersecurity Agents · §4 EvaluationSelected original resource https://arxiv.org/html/2605.14153; sha256:83a1a60bec38561f7d789c0d6161b3934f5df91fbc9c8c6926ef2def526f44c7; archived source capture; exact capture time not independently established
- ExploitBench : A Capability Ladder Benchmark for LLM Cybersecurity Agents · §7 ConclusionSelected original resource https://arxiv.org/html/2605.14153; sha256:83a1a60bec38561f7d789c0d6161b3934f5df91fbc9c8c6926ef2def526f44c7; archived source capture; exact capture time not independently established
Original source charts and materials
Related research questions
References
- [1] Seunghyun Lee, David Brumley. ExploitBench: A Capability Ladder Benchmark for LLM Cybersecurity Agents. 2026-05-13. v1 [arXiv]; Author affiliations on the original paper title page; Figure 1