NUWA Evidence record
Kimi K3
CyberGym · Evidence record
Source snapshot, chart and review scope →
Original source charts and materials
Recorded measurements
- Tasks completed before turn limit
- 78 tasks
78/100; 100 trials
Claude Code via inspect_swe / Inspect model bridge · 100 agent iterations
Assessment provenance
Evaluating Kimi K3 for Misalignment Risks → · External report
Actual evaluation: Neo Research · Model record →
Method and scope
Validated reproduction across a corpus of historical vulnerabilities.
- Evidence type
- Benchmark
- Assessment subject
- AI agent reproducing historical vulnerabilities
- Primary dimension
- Capability
- Evidence snapshot
- 2026-10-29.1
Primary dimension follows the source’s measured construct. Metric-level interpretations may differ.
Source finding
Tasks completed before turn limit: 78 tasks. Conditions are retained from the source.
Limitations
Vulnerability reproduction is narrower than autonomous compromise of deployed systems.
Source transcription is not experimental replication. Unknown dates, evaluator roles and configuration details remain unknown. Source groups are not verified rankings.
Original figure retained without estimating numerical values. Full source tables have not been transcribed.
Related research questions
Cybersecurity / Vulnerability reproduction →
The source directly measures the stated construct under its protocol.
References
- [1] Zhun Wang, Tianneng Shi, et al.. CyberGym: Evaluating AI Agents' Real-World Cybersecurity Capabilities at Scale. 2026-03-24. v3 [arXiv]; Author affiliations on the original paper title page; Figure 2
- [2] Evaluating Kimi K3 for Misalignment Risks. 2026-09-11. September 2026 ; §2.1.3 p.21