NUWA Evidence record
Claude Opus 4.6
CyberGym · Evidence record
Source snapshot, chart and review scope →
Original source charts and materials
Recorded measurements
- CyberGym pass@1
- 44.55%
45/101
Harness not specified · 2M tokens per task
Assessment provenance
GLM-5.2 Risk Evaluation Report → · External report
Actual evaluation: Lyptus Research · Model record →
Method and scope
Validated reproduction across a corpus of historical vulnerabilities.
- Evidence type
- Benchmark
- Assessment subject
- AI agent reproducing historical vulnerabilities
- Primary dimension
- Capability
- Evidence snapshot
- 2026-10-29.1
Primary dimension follows the source’s measured construct. Metric-level interpretations may differ.
Source finding
The source reports 45/101. NUWA calculates the displayed 44.55% from those counts, rounded to two decimal places.
Limitations
Vulnerability reproduction is narrower than autonomous compromise of deployed systems.
122 GLM tasks run; comparison restricted to 101 matched tasks. Nineteen tasks lacked comparator coverage; two tasks excluded as defective. Hard-task selection not representative of all 1507 tasks.
Original figure retained without estimating numerical values. Full source tables have not been transcribed.
Related research questions
Cybersecurity / Vulnerability reproduction →
The source directly measures the stated construct under its protocol.
References
- [1] Zhun Wang, Tianneng Shi, et al.. CyberGym: Evaluating AI Agents' Real-World Cybersecurity Capabilities at Scale. 2026-03-24. v3 [arXiv]; Author affiliations on the original paper title page; Figure 2
- [2] GLM-5.2 Risk Evaluation Report. 2026-08-02. v1.1 · revised 26 August 2026 ; p.8 Fig.3 printed bar label