NUWA Evidence record
Kimi K3
CyberGym · Evidence record
Source snapshot, chart and review scope →
Original source charts and materials
Recorded measurements
- Vulnerabilities successfully reproduced
- 50 tasks
50/100; 100 trials
Claude Code via inspect_swe / Inspect model bridge · 100 agent iterations
Assessment provenance
Evaluating Kimi K3 for Misalignment Risks → · External report
Actual evaluation: Neo Research · Model record →
Method and scope
Validated reproduction across a corpus of historical vulnerabilities.
- Evidence type
- Benchmark
- Assessment subject
- AI agent reproducing historical vulnerabilities
- Primary dimension
- Capability
- Evidence snapshot
- 2026-10-29.1
Primary dimension follows the source’s measured construct. Metric-level interpretations may differ.
Source finding
Vulnerabilities successfully reproduced: 50 tasks. Conditions are retained from the source.
Limitations
Vulnerability reproduction is narrower than autonomous compromise of deployed systems.
50 successes among 78 tasks completed before turn limit; do not silently divide by 78 instead of all 100.
Original figure retained without estimating numerical values. Full source tables have not been transcribed.
Related research questions
Cybersecurity / Vulnerability reproduction →
The source directly measures the stated construct under its protocol.
References
- [1] Zhun Wang, Tianneng Shi, et al.. CyberGym: Evaluating AI Agents' Real-World Cybersecurity Capabilities at Scale. 2026-03-24. v3 [arXiv]; Author affiliations on the original paper title page; Figure 2
- [2] Evaluating Kimi K3 for Misalignment Risks. 2026-09-11. September 2026 ; §2.1.3 p.21