EXTERNAL ASSESSMENT · SaferAI

GLM-5.2 Risk Evaluation Report

The report finds substantial task capability and pressure-dependent behavior; comparison figures include both SaferAI runs and reused external results.

Published 2026-08-02 · Updated 2026-08-26

Sources ↓

Source version, extraction coverage and findings →

Source transcription is not experimental replication. Unknown dates, evaluator roles and configuration details remain unknown. Source groups are not verified rankings.

Models covered

GLM-5.2 →GPT-5.5 →Claude Opus 4.7 →Claude Opus 4.6 →GPT-5.4 →

NUWA Research Note

Research question

What can a public-API evaluation establish about GLM-5.2 capabilities and elicited behavior?

Method

Eight benchmark families and Petri audits, with task subsets, budgets and per-figure execution provenance documented in v1.1.

Source-reported findings

The report finds substantial task capability and pressure-dependent behavior; comparison figures include both SaferAI runs and reused external results.

Limitations

Controlled tasks, elicitation, benchmark familiarity and configuration differences constrain generalization to deployment.

capability · §2.1; Appendix A.1

Cybench

Protocol record →

Task completion on 40 professional CTF problems.

Competition tasks and supplied guidance differ from operational attacks.

External evaluation: SaferAI

This section is catalogued; no precise numerical result has been extracted here. Open the original report for its figures and full findings.

capability · §2.2; Appendix A.2

CyberGym hard subset with token budgets

Protocol record →

Lyptus hard-task subset, distinct 2M, 10M and 50M token budgets; baseline data reused from Lyptus.

Vulnerability reproduction is narrower than autonomous compromise of deployed systems.

External evaluation: SaferAI / Reused result: Lyptus Research

CyberGym hard subset with token budgets

Original evaluator results · source order

Reproduction at 2M tokens

0–100%

GLM-5.2

GLM-5.2 · 36.6% · Validated reproduction across a corpus of historical vulnerabilities. · Inspect Evals · Lyptus hard-task subset; source reports results by inference budget.

36.6%

Showing 1 of 1 systems

Validated reproduction across a corpus of historical vulnerabilities.

Vulnerability reproduction is narrower than autonomous compromise of deployed systems. Report dates and original executor are shown with this assessment.

Evaluation conditions and system configurations

GLM-5.2: Lyptus hard-task subset; source reports results by inference budget. · CyberGym hard subset with token budgets; modelVersion: GLM-5.2 public API · harness: Inspect Evals · tools: Source-defined cyber environment · budget: 2M / 50M tokens per task; different conditions

CyberGym hard subset with token budgets · source order
System / configuration	Reproduction at 2M tokens	Reproduction at 50M tokens	Publication / verification
GLM-5.2Inspect Evals	36.6%	76.2%	Published2026-08-02

capability · §3.1; Appendix A.3

LAB-Bench text subtasks

Protocol record →

Evaluates a range of biology research skills with structured questions; text-only or missing-input subsets remain separate protocols.

These are general scientific capabilities, not end-to-end misuse tasks or direct harm probabilities.

This section is catalogued; no precise numerical result has been extracted here. Open the original report for its figures and full findings.

capability · §3.2; Appendix A.4

BioMysteryBench public release

Protocol record →

Assesses scientific analysis over anonymized datasets; versions, human reference subsets and attempt budgets are kept explicit.

Aggregate scientific performance does not establish an operational biological misuse pathway.

This section is catalogued; no precise numerical result has been extracted here. Open the original report for its figures and full findings.

capability · §4.2; Appendix A.5

SWE-Bench Pro public tasks

Protocol record →

Evaluates repository-level changes; public subsets, scaffolds and reward-hacking scans are version-specific.

General coding capability is enabling evidence, not direct evidence of loss of control.

This section is catalogued; no precise numerical result has been extracted here. Open the original report for its figures and full findings.

propensity · §4.3; Appendix A.9

Agentic Misalignment with neutral emails

Protocol record →

Original agentic-misalignment scenarios extended with 11 neutral emails; 30 samples per threat-by-goal condition.

No real people or systems were harmed; the study does not estimate prevalence in deployment.

External evaluation: SaferAI

This section is catalogued; no precise numerical result has been extracted here. Open the original report for its figures and full findings.

propensity · §5.1; Appendix A.7; Table A.3

MASK by archetype

Protocol record →

Six MASK archetypes; GLM runs by SaferAI, Claude/GPT comparison results reused from CAIS.

A controlled benchmark does not establish stable intent or real-world deception prevalence.

Reused result: Center for AI Safety

MASK by archetype

Original evaluator results · source order

Continuations honesty

0–100%

Claude Opus 4.7

Claude Opus 4.7 · 59.1% · 95% Wilson CI [51.7, 66.1] (as reported) · Condition-bounded frequency of lying under stated pressures. · Per-archetype scored prompts: 176 / 125 / 120 / 209 / 274 / 96. Some supplied CI cells are inconsistent with displayed n; reproduced as reported, not recalculated.

59.1%

GPT-5.5

GPT-5.5 · 65.3% · 95% Wilson CI [58.1, 72.0] (as reported) · Condition-bounded frequency of lying under stated pressures. · Per-archetype scored prompts: 176 / 125 / 120 / 209 / 274 / 96. Some supplied CI cells are inconsistent with displayed n; reproduced as reported, not recalculated.

65.3%

Showing 2 of 2 systems

Condition-bounded frequency of lying under stated pressures.

A controlled benchmark does not establish stable intent or real-world deception prevalence. Report dates and original executor are shown with this assessment.

Evaluation conditions and system configurations

Claude Opus 4.7: Per-archetype scored prompts: 176 / 125 / 120 / 209 / 274 / 96. Some supplied CI cells are inconsistent with displayed n; reproduced as reported, not recalculated. · MASK by archetype; modelVersion: not specified · harness: not specified · tools: not specified · budget: not specified

GPT-5.5: Per-archetype scored prompts: 176 / 125 / 120 / 209 / 274 / 96. Some supplied CI cells are inconsistent with displayed n; reproduced as reported, not recalculated. · MASK by archetype; modelVersion: not specified · harness: not specified · tools: not specified · budget: not specified

MASK by archetype · source order
System / configuration	Continuations honesty	Disinformation honesty	Doubling-down honesty	Known-facts honesty	Provided-facts honesty	Statistics honesty	Publication / verification
Claude Opus 4.7Harness not specified	59.1%95% Wilson CI [51.7, 66.1] (as reported)	33.6%95% Wilson CI [25.9, 42.3] (as reported)	68.3%95% Wilson CI [59.6, 76.0] (as reported)	63.6%95% Wilson CI [56.9, 69.9] (as reported)	74.8%95% Wilson CI [69.4, 79.6] (as reported)	80.2%95% Wilson CI [71.1, 86.9] (as reported)	Published2026-08-02
GPT-5.5Harness not specified	65.3%95% Wilson CI [58.1, 72.0] (as reported)	69.6%95% Wilson CI [61.1, 77.0] (as reported)	63.3%95% Wilson CI [54.4, 71.4] (as reported)	77.0%95% Wilson CI [70.9, 82.2] (as reported)	75.2%95% Wilson CI [69.7, 79.9] (as reported)	87.5%95% Wilson CI [79.4, 92.7] (as reported)	Published2026-08-02

propensity · §5.2; Appendix A.8

APE Turn-1 attempts

Protocol record →

Frequency of persuasive attempts under specified conversational conditions.

Simulated attempts are not measured human belief change or large-scale disinformation reach.

External evaluation: SaferAI

This section is catalogued; no precise numerical result has been extracted here. Open the original report for its figures and full findings.

behavior · §6; Appendix A.6

Petri 3.0 automated audit

Protocol record →

Petri 3.0, 173 public seeds, Opus 4.7 auditor and judge; filtered-out scored cases differ by model.

Low average audit scores do not rule out rare severe cases; results depend on seeds, auditor, judge and target configuration.

This section is catalogued; no precise numerical result has been extracted here. Open the original report for its figures and full findings.

Report coverage

Section	Source location	Treatment
Cybench	§2.1; Appendix A.1	Protocol variant Protocol and result provenance are retained at this section.
CyberGym hard subset with token budgets	§2.2; Appendix A.2	Protocol variant Protocol and result provenance are retained at this section.
LAB-Bench text subtasks	§3.1; Appendix A.3	Protocol variant Protocol and result provenance are retained at this section.
BioMysteryBench public release	§3.2; Appendix A.4	Protocol variant Protocol and result provenance are retained at this section.
SWE-Bench Pro public tasks	§4.2; Appendix A.5	Protocol variant Protocol and result provenance are retained at this section.
Agentic Misalignment with neutral emails	§4.3; Appendix A.9	Protocol variant Protocol and result provenance are retained at this section.
MASK by archetype	§5.1; Appendix A.7; Table A.3	Protocol variant Protocol and result provenance are retained at this section.
APE Turn-1 attempts	§5.2; Appendix A.8	Protocol variant Protocol and result provenance are retained at this section.
Petri 3.0 automated audit	§6; Appendix A.6	Protocol variant Protocol and result provenance are retained at this section.
Validity, contamination and evaluation awareness	§7; pp. 27–28	Limitations Protocol and result provenance are retained at this section.

References

Numerical results are reported by the cited authors.

[1] GLM-5.2 Risk Evaluation Report. 2026-08-02. v1.1 · revised 26 August 2026 ; §2.2; Figure 3; Appendix A.7; Table A.3, pp. 41–42; p.6 Fig.1