Risk Evaluation · Benchmark

EVMbench

NUWA research note · independent review pending

First published: Publication date not verifiedSource checked: 2026-10-25

Original source ↗Cite this record ↓Cybersecurity →

NUWA Research Note

Research question Design and assumptions Source findings NUWA interpretation Limits and contrary evidence Research and decision relevance Institutions and roles Original resources

Model-assisted reading · Selected sections of the original full text or official article; a complete section-by-section audit remains pending.

At a glance

NUWA interpretation: detect, patch and exploit scores measure distinct capabilities.

Research question

Can agents detect, patch and exploit historical smart-contract vulnerabilities?

Design and assumptions

117 curated vulnerabilities from forty audits support separate modes; isolated-chain execution and deterministic transaction replay validate exploit tasks.

Source findings

GPT-5.3-Codex via Codex CLI scored 71.0% in exploit mode, versus GPT-5’s 33.3%.

NUWA interpretation

NUWA interpretation: detect, patch and exploit scores measure distinct capabilities.

Limits and contrary evidence

Known-ground-truth detection misses novel issues; sequential replay excludes timing-dependent behavior, and local single-chain tests limit realism.

Research and decision relevance

Model recommendation: cite mode and harness conditions; do not treat exploit scores as live-network compromise rates.

Precise source locations

EVMbench · Introducing EVMbench: capability modes (official article paragraphs 31–44)Selected original resource https://openai.com/index/introducing-evmbench/; sha256:0d7b334d1452cbe75c15509ed172d187c3c9e70ce186f39678761a9e9ac8ddaa; web-tool-extracted-response; not original HTML
EVMbench · Limitations (paragraphs 45–49)Selected original resource https://openai.com/index/introducing-evmbench/; sha256:0d7b334d1452cbe75c15509ed172d187c3c9e70ce186f39678761a9e9ac8ddaa; web-tool-extracted-response; not original HTML

Source checking, extraction, editorial approval and replication are recorded separately. Methodology →

Institutions and roles

Affiliation not specified

Evaluation protocols

EVMbench →

Cite and reuse

Cite the original publication for its findings. Cite this NUWA record separately for its explanation and retained snapshot.

BibTeX ↓RIS ↓Record + source metadata · JSON ↓Reading coverage · JSON Suggest a correction

Original publication · BibTeX

@misc{evmbench,
  title = {EVMbench},
  url = {https://openai.com/index/introducing-evmbench/},
  version = {2026-10-25 source capture},
  note = {2026-10-25 source capture}
}

NUWA record · BibTeX

@misc{evmbench-nuwa,
  title = {EVMbench — NUWA research record},
  url = {https://whitzard.tech/en/nuwa/whitzard-index/work/evmbench/},
  version = {2026-10-29.1},
  organization = {NUWA Frontier AI Safety Lab},
  howpublished = {Retained note edition: https://whitzard.tech/nuwa/whitzard-index/data/record-editions/2026-10-29.1--evmbench.json},
  note = {NUWA research note. Source-reading scope and independent review status are stated on the record. Snapshot 2026-10-29.1. Retained note edition: https://whitzard.tech/nuwa/whitzard-index/data/record-editions/2026-10-29.1--evmbench.json.}
}

Evidence snapshot 2026-10-29.1 · Index updates · RSS

References

[1] EVMbench ; Introducing EVMbench: capability modes (official article paragraphs 31–44); Limitations (paragraphs 45–49)