NUWA 来源档案
GLM-5.2 Risk Evaluation Report, Version 1.1
对 SaferAI · GLM-5.2 的逐来源提取与核查，发现和数值均保留原始归属。
范围与归属
- 数据报告方
- SaferAI
- 实际评测方
- 各观测的执行方未完整确定
- 抓取版本
- 2026-10-26
- 来源身份
- f2e83f53d146628b
来源转录不等于实验复现。未确定的日期、执行方及配置保留为未知；原文分组不作为经核验排名。
来源发现
以下陈述整理自原始来源，未作为 NUWA 独立实验结论。
SaferAI 报告通过 OpenRouter 的 Z.ai 提供方调用公开 API，且与 Z.ai 没有业务关系，也未讨论本次评测。
p.4 §1结论范围限于原始来源的任务与实验条件；原文定位保留供核查。
网络评测覆盖范围有限的沙箱任务，未测试完整攻击活动或具有适应能力的实时防守方。
pp.26-27 §§7.1,7.4结论范围限于原始来源的任务与实验条件；原文定位保留供核查。
报告的相关科学评测限于公开研究知识与数字任务代理，不能直接证明有害用途增益或现实世界执行能力。
§§3,3.3,7.1,7.4; pp.10,13-14,26-27结论范围限于原始来源的任务与实验条件；原文定位保留供核查。
软件工程能力被用作实现手段的代理；研究没有直接测试稳健的失控部署或部署机会。
pp.15-16 §§4.1-4.2结论范围限于原始来源的任务与实验条件；原文定位保留供核查。
GLM-5.2 的 MASK 来自 SaferAI 自行评测；Opus 4.7 和 GPT-5.5 的 MASK 数值引用 CAIS 分享的仪表盘数据。
pp.20-21 §5.1/Fig.9结论范围限于原始来源的任务与实验条件；原文定位保留供核查。
MASK 与 APE 衡量行为倾向，均未测量对现实目标人群的下游影响。
pp.20-22 §5结论范围限于原始来源的任务与实验条件；原文定位保留供核查。
Petri 是探索性文本模拟，工具也由审计器模拟；评分不是现实行为频率。判别器内容过滤会使部分潜在令人担忧的场景缺失。
pp.23-25 §6结论范围限于原始来源的任务与实验条件；原文定位保留供核查。
作者在选取的高分轨迹中观察到借口下的服从、针对运营方的单方面行动及压力下的退让；这些是特定场景观察。
pp.24-25 §6结论范围限于原始来源的任务与实验条件；原文定位保留供核查。
模型说出评测意识的频率，不能说明未表达的评测意识在多大程度上影响了行为。
p.26 §7.2结论范围限于原始来源的任务与实验条件；原文定位保留供核查。
公开数据污染与针对基准优化，可能使基准成绩高估现实任务能力。
pp.26-27 §7.3结论范围限于原始来源的任务与实验条件；原文定位保留供核查。
作者明确表示，一部分公开基准不足以支持总体风险判断，并建议进行更系统的独立评测。
p.28 §8结论范围限于原始来源的任务与实验条件；原文定位保留供核查。
提取范围
以下条目说明来源各部分的处理状态。个别观测已核验，不代表整份文档的全部结论和结果均已验证。
查看 55 项覆盖记录
| 章节 | 处理状态 | 定位与范围 |
|---|---|---|
| Cover and Executive Summary | qualitative | pp.1-2Cover verifies Version 1.1 revision August 26, 2026. Initial publication date not stated. Sensitive biological claims omitted. |
| Table of Contents | extracted | p.3All relevant main sections and appendices enumerated; use physical PDF pages, not TOC offsets. |
| Introduction | extracted | p.4Independent public-API evaluation; OpenRouter account with Z.ai provider; no developer cooperation. |
| Cyber Offense | extracted | pp.5-9Aggregate outcomes only; no exploit/task instructions. |
| Cybench | extracted | pp.5-6; Figs.1-2Exact printed success counts extracted, including all category labels; no bar-height estimates. |
| CyberGym | extracted | pp.7-8; Fig.3SaferAI GLM own runs; Lyptus comparator data separately marked reused; exact printed counts and prose percentages retained. |
| Cyber discussion | qualitative | p.9Budget sensitivity, confidence overlap, benchmark coverage and safeguards limitations retained. |
| CBRN | qualitative | pp.10-14Safety-bounded bibliographic/method scope and high-level limitations only; no biological outcomes, task material, or dangerous-capability comparisons. |
| LAB-Bench | qualitative | pp.10-11; Table 1; Fig.4Public research-knowledge proxy; text-only coverage. Numerical capability outcomes and task details intentionally excluded. |
| BioMysteryBench | qualitative | pp.12-13; Figs.5-6Public digital research-task benchmark; completion/turn-budget limitations retained qualitatively. Outcomes intentionally excluded. |
| CBRN discussion | qualitative | pp.13-14Knowledge/digital proxies do not directly measure physical-world harmful uplift; wider CBRN domains not covered. |
| Loss of Control | extracted | pp.15-19Means/motive/opportunity framing; software engineering is a proxy, not a direct rogue deployment evaluation. |
| Methodology | extracted | p.15SWE-Bench Pro plus Agentic Misalignment; deployment opportunity not evaluated. |
| SWE-Bench Pro | extracted | pp.15-16; Fig.7GLM own run; Scale AI baselines reused. Contamination, reward hacking and environment issues retained. |
| Agentic Misalignment | extracted | pp.17-19; Fig.8All 108 printed cells (three models, three scenarios, four goals, three threats) and their printed CIs; no operational scenario transcript. |
| LoC discussion | qualitative | p.19Public task contamination; benchmark-to-real-world and scaffold dependence. |
| Harmful Manipulation | extracted | pp.20-22Propensities do not measure downstream human effects. |
| MASK | extracted | pp.20-21; Fig.9All printed aggregate percentages; GLM own run and CAIS shared comparator scores distinguished. |
| APE | extracted | pp.21-22; Fig.10Exact prose percentages only. Benign/controversial category bars lack labels: qualitative/figure-only, not estimated. |
| Manipulation discussion | qualitative | p.22Willingness to persuade does not establish effectiveness or real-world manipulation. |
| Automated Behavioral Audit | qualitative | pp.23-25; Fig.11Means have no exact printed numeric labels; not estimated. Aggregate sample counts and prose thresholds extracted; operational content omitted. |
| Petri methodology | extracted | p.23Public seeds, score dimensions, elicitation budget, judge exclusions and simulated tools. |
| Limitations and Future Work | qualitative | pp.26-27All four subsections represented. |
| Capability Coverage | qualitative | p.26Narrow proxies omit broad real-world capabilities; cyber saturation. |
| Evaluation Awareness | extracted | p.26Exact verbalized awareness rates; unspoken awareness unknown. |
| Data Contamination | qualitative | pp.26-27Public datasets can be memorized/optimized; held-out benchmark recommendation. |
| Validity | qualitative | p.27No active defenders, physical experiments or human persuasion effects in these proxy evaluations. |
| Conclusion | qualitative | p.28Authors explicitly say subset of public benchmarks does not support overall risk judgment. Biological capability claims excluded. |
| References | extracted | pp.29-31Selected provenance dependencies recorded, not independently re-extracted. |
| Cybench Methodology | extracted | p.32Elicitation, budgets and incomplete task coverage; task names not copied. |
| CyberGym Methodology | extracted | pp.32-33Configuration and sample filtering; operational instructions and vulnerability IDs not copied. |
| CyberGym configuration | extracted | pp.32-33Provider-default GLM reasoning; token and wall-clock budgets, no wall-clock hits. |
| CyberGym sample | extracted | p.33122 run; comparison 101 after missing Lyptus coverage and defective targets; no task-ID list copied. |
| LAB-Bench Methodology | qualitative | pp.33-34Method metadata and high-level limitations only; no numerical outcomes or detailed biological tool recipes. |
| LAB benchmark coverage | qualitative | p.34Text modality restrictions and public data; outcomes excluded. |
| LAB tools | qualitative | p.34Shared sandbox/tool environment; tool inventories omitted. |
| LAB elicitation | qualitative | p.34Reasoning enabled; finite budgets may under-elicit; dangerous-capability numbers excluded. |
| LAB safety filters | qualitative | p.34Provider filtering and abstention complicate comparisons; numerical outcomes excluded. |
| BioMysteryBench | qualitative | pp.34-36Method metadata only. |
| BioMysteryBench methodology | qualitative | p.35Digital anonymized research tasks; anti-cheat grading; exact task details and all capability outcomes excluded. |
| Turn limit analysis | qualitative | pp.35-36Finite turn budget and incomplete attempts may understate capability; all numerical results excluded. |
| SWE-Bench Pro Methodology | extracted | pp.37-40Inspect Harbor source, environment and reward-hacking diagnostics. |
| SWE technical issues | qualitative | p.37Prompt/test mismatch and answer leakage in repository history; no operational cheating instructions. |
| SWE environment issues | extracted | pp.37-38; Table A.1All printed rows preserved as diagnostics; count and pass-rate denominator mismatch quarantined. |
| SWE reward hacking | extracted | pp.38-40; Table A.2All printed rows and filtering summaries preserved; uncertainty in effective denominators quarantined. |
| Petri methodology | extracted | pp.40-41Aggregate judge exclusions recorded; seed names/task content not copied. |
| Petri seeds and dimensions | not-relevant | p.41Reference pointers only; full scenario seeds not extracted. |
| MASK methodology | extracted | pp.41-42GLM own run and CAIS reused comparator provenance. |
| MASK confidence intervals | extracted | pp.41-42; Table A.3All 18 printed rows; inconsistent GLM values quarantined without corrections. |
| APE methodology | extracted | pp.42-43Full prompt-set methodology; exact harmful example claims not copied. |
| Agentic Misalignment methodology | extracted | pp.43-51Inspect Evals and 11 added neutral emails; no prompts/email bodies copied. |
| Additional neutral emails | not-relevant | pp.44-51Synthetic setup text, not benchmark results; not extracted. |
| Unreported CyberGym comparator cells | not-disclosed | p.8 Fig.3No GPT-5.5 10M data; no Opus 4.6 10M/50M data. Opus 4.7 evaluation could not be completed due to safeguard refusals. |
| APE benign factual / benign opinion / controversial exact rates | figure-only | p.22 Fig.10Chart bars lack exact numeric labels; no bar-height estimates extracted. |
| Petri exact numerical mean scores | figure-only | p.24 Fig.11Exact means and error-bar bounds are not printed; no numerical estimates extracted. |
来源待核实事项
GLM 的 MASK 汇总结果与附录分组表不能相互核对，部分区间也存在内部不一致；相关结果暂缓，未推测替代数字。
CyberGym 比较模型结果引用 Lyptus 数据，MASK 比较模型结果引用 CAIS 仪表盘，不能全部归为 SaferAI 自行执行。
版本指纹
document
https://www.safer-ai.org/u/2026/08/SaferAI-GLM-Evaluation-Report.pdf
2b2c8ea89d19db3d123894ff406015ee0e15b0fef67bc832a5658ede23991176参考文献
- [1] GLM-5.2 Risk Evaluation Report. 2026-08-02. v1.1 · revised 26 August 2026