WhitzardNUWA Lab ↗
NUWANUWA INDEX

Search NUWA Index

Research · Benchmarks · Findings · Perspectives

中文
← NUWA Index

Evidence record

GLM-5.2

CyberGym · Evidence record

University of California, Berkeley / arXiv:2506.02548v3 / Capability

Sources and provenance ↓ Read the research narrative →

Original source charts and materials

CyberGym hard subset with token budgets

Original evaluator results · source order

Reproduction at 2M tokens

0–100%
GLM-5.2
36.6%
Showing 1 of 1 systems

Validated reproduction across a corpus of historical vulnerabilities.

Vulnerability reproduction is narrower than autonomous compromise of deployed systems. Report dates and original executor are shown with this assessment.

Evaluation conditions and system configurations
GLM-5.2
Lyptus hard-task subset; source reports results by inference budget. · CyberGym hard subset with token budgets
modelVersion: GLM-5.2 public API · harness: Inspect Evals · tools: Source-defined cyber environment · budget: 2M / 50M tokens per task; different conditions
CyberGym hard subset with token budgets · source order
System / configurationReproduction at 2M tokensReproduction at 50M tokensPublication / verification
GLM-5.2Inspect Evals36.6%76.2%Published2026-08-02

Assessment provenance

GLM-5.2 Risk Evaluation Report → · External report

Actual evaluation: SaferAI · Model record →

Method and scope

Validated reproduction across a corpus of historical vulnerabilities.

Evidence type
Benchmark
Assessment subject
AI agent reproducing historical vulnerabilities
Primary dimension
Capability
Evidence snapshot
2026-10-29.1

Primary dimension follows the source’s measured construct. Metric-level interpretations may differ.

Source finding

Source-reported measurements in §2.2; Figure 3. Transcription is not independent experimental replication.

Limitations

Vulnerability reproduction is narrower than autonomous compromise of deployed systems.

Vulnerability reproduction is narrower than autonomous compromise of deployed systems.

Original figure retained without estimating numerical values. Full source tables have not been transcribed.

Related research questions

References

  1. [1] Zhun Wang, Tianneng Shi, et al.. CyberGym: Evaluating AI Agents' Real-World Cybersecurity Capabilities at Scale. 2026-03-24. v3 [arXiv]; Author affiliations on the original paper title page; Figure 2
  2. [2] GLM-5.2 Risk Evaluation Report. 2026-08-02. v1.1 · revised 26 August 2026 ; §2.2; Figure 3