{"schemaVersion":"nuwa-current-record-1","version":"2026-10-29.1","publication":"published","work":{"id":"exploitbench","title":"ExploitBench: A Capability Ladder Benchmark for LLM Cybersecurity Agents","aliases":["ExploitBench"],"authors":["Seunghyun Lee","David Brumley"],"primaryFunction":"evaluation","secondaryFunctions":[],"kind":"benchmark","sourceId":"exploitbench-publication","attributionSourceId":"exploitbench-publication-author-source","originalUrl":"https://arxiv.org/abs/2605.14153","publicationDate":"2026-05-13","updatedOn":"2026-10-25","areas":["cybersecurity"],"scopeNote":{"en":"The main dimension follows the task outcome in the original protocol. Topic relevance is documented in the parent research record; no unreviewed question-level links are inferred.","zh":"主维度依据原协议的任务结果确定。主题关联见研究档案，不推断尚未审阅的问题级关联。"},"protocolIds":["exploitbench"],"contributions":[{"organizationId":"cmu","role":"research","sourceId":"exploitbench-publication-author-source","locator":"Author affiliations on the original paper title page"},{"organizationId":"bugcrowd","role":"research","sourceId":"exploitbench-publication-author-source","locator":"Author affiliations on the original paper title page"}],"summary":{"en":"Separates exploit development into capability levels, distinguishing bug triggering from reusable exploitation primitives and control.","zh":"将漏洞利用拆分为能力层级，区分触发缺陷、构造利用原语和获取控制。"},"note":{"question":null,"method":null,"finding":null,"supports":null,"limits":null,"connections":null},"resources":[],"noteStatus":{"question":"limited","method":"limited","finding":"limited","supports":"limited","limits":"limited","connections":"limited"},"noteAvailability":{"en":"A substantive NUWA research note is not yet available. The repeated draft sections were removed pending review of the original methods, findings and limitations; use the original publication for the research account.","zh":"实质性的 NUWA 研究笔记尚未完成。重复草稿章节已移除，待核查原始方法、发现与限制；研究内容请以原始出版物为准。"},"noteSourceLocator":"No complete section-level method/results/limitations review completed for this note; cached original page and any chart audit retained as discovery evidence.","tags":[]},"reading":{"id":"exploitbench","title":"ExploitBench: A Capability Ladder Benchmark for LLM Cybersecurity Agents","en":{"question":"Where do agents stop between triggering a known bug and achieving demonstrated security impact?","design":"Forty-one V8 bugs are graded with sixteen deterministic capability flags, differential checks and per-run randomized verification.","findings":["The public-model panel rarely crossed the reported Tier-3-to-Tier-2 boundary; GPT-5.5 did so on one bug.","Mythos Preview crossed more consistently, distinguishing a benchmark possibility from public-model behavior (Conclusion)."],"interpretation":"NUWA interpretation: graded intermediate capability reveals information hidden by binary crash success.","limits":"One-bug crossings are existence proofs; this V8 population and bounded scaffold do not estimate deployed attack rates.","decisionRelevance":"Model recommendation: report achieved capability tier and verification conditions rather than labeling every crash an exploit.","takeaway":"NUWA interpretation: graded intermediate capability reveals information hidden by binary crash success.","connections":[],"scoring":"Sixteen deterministically verified capability flags on forty-one V8 bugs.","conditions":"Binary crash is not final impact; per-run randomized verification and differential checks.","statistics":"The public-model panel rarely crossed the reported Tier-3-to-Tier-2 boundary; GPT-5.5 did so on one bug.","usage":"Model recommendation: report achieved capability tier and verification conditions rather than labeling every crash an exploit."},"zh":{"question":"从触发已知缺陷到证实安全影响，智能体停在哪一步？","design":"41个V8缺陷以16个确定性能力标志、差分检查和每次随机验证判分。","findings":["公开模型很少跨越所述Tier3到Tier2边界，GPT-5.5仅在一个缺陷跨越。","Mythos Preview更稳定，说明基准可达性与公开模型表现有别（结论）。"],"interpretation":"NUWA 解读：中间能力等级能揭示二元崩溃指标隐藏的信息。","limits":"单例跨越只证明存在性；V8样本和有限框架不能估计部署攻击率。","decisionRelevance":"模型建议：报告能力等级与验证条件，不将所有崩溃称为完整利用。","takeaway":"NUWA 解读：中间能力等级能揭示二元崩溃指标隐藏的信息。","connections":[],"scoring":"41个V8缺陷以16个确定性验证能力标志判分。","conditions":"二元崩溃不是最终影响，含逐次随机验证与差分检查。","statistics":"公开模型很少跨越所述Tier3到Tier2边界，GPT-5.5仅在一个缺陷跨越。","usage":"模型建议：报告能力等级与验证条件，不将所有崩溃称为完整利用。"},"assertionPolicy":{"question":"source-research-question","design":"source-finding","findings":"source-finding","interpretation":"nuwa-interpretation","limits":"source-limit-or-explicit-editorial-scope-limit","connections":"source-supported-relationship-only","decisionRelevance":"model-recommendation-not-source-conclusion"},"sourceRefs":[{"sourceId":"exploitbench-publication-author-source","url":"https://arxiv.org/html/2605.14153","locator":"Abstract","version":"Selected original resource https://arxiv.org/html/2605.14153; sha256:83a1a60bec38561f7d789c0d6161b3934f5df91fbc9c8c6926ef2def526f44c7; archived source capture; exact capture time not independently established","artifactSha256":"83a1a60bec38561f7d789c0d6161b3934f5df91fbc9c8c6926ef2def526f44c7"},{"sourceId":"exploitbench-publication-author-source","url":"https://arxiv.org/html/2605.14153","locator":"§2 Design","version":"Selected original resource https://arxiv.org/html/2605.14153; sha256:83a1a60bec38561f7d789c0d6161b3934f5df91fbc9c8c6926ef2def526f44c7; archived source capture; exact capture time not independently established","artifactSha256":"83a1a60bec38561f7d789c0d6161b3934f5df91fbc9c8c6926ef2def526f44c7"},{"sourceId":"exploitbench-publication-author-source","url":"https://arxiv.org/html/2605.14153","locator":"§4 Evaluation","version":"Selected original resource https://arxiv.org/html/2605.14153; sha256:83a1a60bec38561f7d789c0d6161b3934f5df91fbc9c8c6926ef2def526f44c7; archived source capture; exact capture time not independently established","artifactSha256":"83a1a60bec38561f7d789c0d6161b3934f5df91fbc9c8c6926ef2def526f44c7"},{"sourceId":"exploitbench-publication-author-source","url":"https://arxiv.org/html/2605.14153","locator":"§7 Conclusion","version":"Selected original resource https://arxiv.org/html/2605.14153; sha256:83a1a60bec38561f7d789c0d6161b3934f5df91fbc9c8c6926ef2def526f44c7; archived source capture; exact capture time not independently established","artifactSha256":"83a1a60bec38561f7d789c0d6161b3934f5df91fbc9c8c6926ef2def526f44c7"}],"publication":{"state":"published","publishedOn":"2026-10-29","authorizationReference":"user-authorized-global-index-publication-2026-10-29","reviewedCandidateId":"local-6efe31bc4a0c0f37"},"review":{"state":"published-note","sourceScope":"full-text","availableScope":null,"readingCoverage":"selected-source-sections; not a complete section-by-section paper audit","completePaperAudit":false,"independentEvidenceReview":"pending-human-review","bilingualEditorialReview":"pending-human-review","authorship":"model-assisted-research-note","preparedFingerprint":"61f5c841cbc9c1b78a0e8c0056b343039aa1ea2971ed5cb810e8ebe4abdca34d"}},"sources":[{"id":"exploitbench-publication","title":"ExploitBench: A Capability Ladder Benchmark for LLM Cybersecurity Agents","url":"https://arxiv.org/abs/2605.14153","version":"arXiv:2605.14153v1","publicationDate":"2026-05-13","datePrecision":"day","type":"paper","verifiedOn":"2026-10-25","documentId":"arxiv:2605.14153","authors":["Seunghyun Lee","David Brumley"],"aliases":["https://doi.org/10.48550/arXiv.2605.14153","https://arxiv.org/abs/2605.14153","https://arxiv.org/pdf/2605.14153"]}]}