NUWA Evidence record
Claude Sonnet 3.7 · Anthropic Agent
CyberGym · Evidence record
Source snapshot, chart and review scope →
Original source charts and materials
Recorded measurements
- Success rate
- 47.2%
Per-result sample size not disclosed
Anthropic Agent · Budget not specified
Method and scope
Validated reproduction across a corpus of historical vulnerabilities.
- Evidence type
- Benchmark
- Assessment subject
- AI agent reproducing historical vulnerabilities
- Primary dimension
- Capability
- Evidence snapshot
- 2026-10-29.1
Primary dimension follows the source’s measured construct. Metric-level interpretations may differ.
Source finding
The source reports the fraction 0.472. The displayed 47.2% follows multiplication by 100 and rounding to 1 decimal place(s).
Limitations
Vulnerability reproduction is narrower than autonomous compromise of deployed systems.
The score_10 field name does not specify ten trials. Unknown budgets and per-row denominators remain unknown. Team-submitted leaderboard result; no per-row execution receipt or independent reproduction in this extraction. Row denominator, benchmark revision/subset, actual execution date and per-row resource budgets are not disclosed. Source label and URL do not establish the actual executor, developer or submitter. Thirty attempts per instance; omitted by the current single-trial UI selector. Do not compare as one-trial results.
Original figure retained without estimating numerical values. Full source tables have not been transcribed.
Related research questions
Cybersecurity / Vulnerability reproduction →
The source directly measures the stated construct under its protocol.
References
- [1] Zhun Wang, Tianneng Shi, et al.. CyberGym: Evaluating AI Agents' Real-World Cybersecurity Capabilities at Scale. 2026-03-24. v3 [arXiv]; Author affiliations on the original paper title page; Figure 2
- [2] CyberGym official leaderboard and machine-data extraction ; /level1/66/score_10