NUWA Evidence record
DeepSeek-V4-Flash · Ostorlab Neutron
CyberGym · Evidence record
Source snapshot, chart and review scope →
Original source charts and materials
Recorded measurements
- Success rate
- 97.1%
Per-result sample size not disclosed
Ostorlab Neutron · Budget not specified
Assessment provenance
CyberGym official snapshot → · External report
Actual evaluation: · Model record →
Method and scope
Validated reproduction across a corpus of historical vulnerabilities.
- Evidence type
- Benchmark
- Assessment subject
- AI agent reproducing historical vulnerabilities
- Primary dimension
- Capability
- Evidence snapshot
- 2026-10-29.1
Primary dimension follows the source’s measured construct. Metric-level interpretations may differ.
Source finding
The source reports the fraction 0.9708. The displayed 97.1% follows multiplication by 100 and rounding to 1 decimal place(s).
Limitations
Vulnerability reproduction is narrower than autonomous compromise of deployed systems.
The score_10 field name does not specify ten trials. Unknown budgets and per-row denominators remain unknown. Team-submitted leaderboard result; no per-row execution receipt or independent reproduction in this extraction. Row denominator, benchmark revision/subset, actual execution date and per-row resource budgets are not disclosed. Source label and URL do not establish the actual executor, developer or submitter.
Original figure retained without estimating numerical values. Full source tables have not been transcribed.
Related research questions
Cybersecurity / Vulnerability reproduction →
The source directly measures the stated construct under its protocol.
References
- [1] Zhun Wang, Tianneng Shi, et al.. CyberGym: Evaluating AI Agents' Real-World Cybersecurity Capabilities at Scale. 2026-03-24. v3 [arXiv]; Author affiliations on the original paper title page; Figure 2
- [2] CyberGym official leaderboard and machine-data extraction ; /level1/13/score_10