NUWA Evidence record
GPT-4.1 · OpenHands
CyberGym · Evidence record
Source snapshot, chart and review scope →
Original source charts and materials
Recorded measurements
- Success rate
- 17.1%
Per-result sample size not disclosed
OpenHands · Budget not specified
Method and scope
Validated reproduction across a corpus of historical vulnerabilities.
- Evidence type
- Benchmark
- Assessment subject
- AI agent reproducing historical vulnerabilities
- Primary dimension
- Capability
- Evidence snapshot
- 2026-10-29.1
Primary dimension follows the source’s measured construct. Metric-level interpretations may differ.
Source finding
The source reports the fraction 0.1712010617120106. The displayed 17.1% follows multiplication by 100 and rounding to 1 decimal place(s).
Limitations
Vulnerability reproduction is narrower than autonomous compromise of deployed systems.
The score_10 field name does not specify ten trials. Unknown budgets and per-row denominators remain unknown. Team-submitted leaderboard result; no per-row execution receipt or independent reproduction in this extraction. Row denominator, benchmark revision/subset, actual execution date and per-row resource budgets are not disclosed. Source label and URL do not establish the actual executor, developer or submitter. Not shown by the current Level1 leaderboard selector; task information differs by level. Trials are absent and remain unknown.
Original figure retained without estimating numerical values. Full source tables have not been transcribed.
Related research questions
Cybersecurity / Vulnerability reproduction →
The source directly measures the stated construct under its protocol.
References
- [1] Zhun Wang, Tianneng Shi, et al.. CyberGym: Evaluating AI Agents' Real-World Cybersecurity Capabilities at Scale. 2026-03-24. v3 [arXiv]; Author affiliations on the original paper title page; Figure 2
- [2] CyberGym official leaderboard and machine-data extraction ; /level3/0/score_10