NUWA 来源档案
Evaluating DeepSeek v4 Pro for Frontier Risks
对 Neo Research · DeepSeek V4 Pro 的逐来源提取与核查，发现和数值均保留原始归属。
范围与归属
- 数据报告方
- Neo Research
- 实际评测方
- 各观测的执行方未完整确定
- 抓取版本
- 2026-10-26
- 来源身份
- 267c0a21feca7aeb
来源转录不等于实验复现。未确定的日期、执行方及配置保留为未知；原文分组不作为经核验排名。
来源发现
以下陈述整理自原始来源，未作为 NUWA 独立实验结论。
作者认为当前网络能力低于其关切阈值，同时说明《实践准则》本身没有规定网络能力的数值阈值。这是作者判断，不是监管认证。
pp.7,10结论范围限于原始来源的任务与实验条件；原文定位保留供核查。
作者认为公开知识与研究流程基准只是现实用途增益的薄弱代理，因此不作确定的相关滥用风险结论，并建议进行针对性评测或人类增益研究。
pp.11–13结论范围限于原始来源的任务与实验条件；原文定位保留供核查。
强制二元选择的说服任务可能掩盖自由续写条件下较高的说服尝试倾向；尝试率不等于说服成功率。
pp.14–18结论范围限于原始来源的任务与实验条件；原文定位保留供核查。
不当行为高度依赖场景：当评测者提供动机或施压时，故意有害行动与欺骗增多；这些压力测试不能证明自主谋划或现实发生频率。
pp.21–32结论范围限于原始来源的任务与实验条件；原文定位保留供核查。
来源的独立与内部测试提示，安全防护在对抗压力下较脆弱，Pro 与 Flash 存在相关弱点；这不是所有使用情境下的统一失败率。
pp.33–38结论范围限于原始来源的任务与实验条件；原文定位保留供核查。
言语化评测意识随场景与监测器变化，不能据此证明意识引起了行为变化；模型也可能从训练数据认出基准。
pp.39–44结论范围限于原始来源的任务与实验条件；原文定位保留供核查。
自动审计结果取决于审计器、场景选择、对话预算与判别标准；选取高分种子不代表总体发生率。
pp.45–53结论范围限于原始来源的任务与实验条件；原文定位保留供核查。
在测试范围内，二元判别结果较稳定，而开放式序数评分的跨判别器分歧更大；量表定义与高方差限制了结论。
pp.47–53结论范围限于原始来源的任务与实验条件；原文定位保留供核查。
提取范围
以下条目说明来源各部分的处理状态。个别观测已核验，不代表整份文档的全部结论和结果均已验证。
查看 36 项覆盖记录
| 章节 | 处理状态 | 定位与范围 |
|---|---|---|
| Title / Executive summary / Introduction | qualitative | pp.1–6Report dated 29 May 2026; independent Neo evaluation without privileged access. v1.1 URL does not establish revision date. |
| 1 Cyber — Cybench | extracted | pp.7–8Exact chart labels retained; comparator values explicitly reused; no CI endpoint estimates. |
| 1 Cyber — EVMbench | extracted | pp.8–9All 16 printed figure values and n=3 labels retained. Comparator executor/reuse status unclear, quarantined. |
| 1 Cyber — WMDP-Cyber | extracted | pp.9–10Complete five-model knowledge table; controls reused from Muse Spark report. |
| 2 CBRN — LAB-Bench | qualitative | pp.11–12Metadata only: research-workflow MCQ proxy; no real wet-lab uplift established; image tasks unsupported at testing. Dangerous-capability numbers intentionally excluded. |
| 2 CBRN — WMDP Chem/Bio | qualitative | p.12Knowledge-only evaluation; saturation limits inference. Biological/chemical numerical comparisons excluded. |
| 2 CBRN — FORTRESS / conclusion | qualitative | p.13Adversarial CBRNE and benign-overrefusal methodology; ensemble-majority judge with refusal-related grading bias. Numerical CBRN comparisons excluded. |
| 3.1 PersuSafety | extracted | pp.14–15Exact counts/rates from text; unlabelled stacked-chart cells left figure-only; reused older controls flagged. |
| 3.1 Attempt-to-Persuade | extracted | pp.15–16All five subject rows; refusal/attempt counts preserved in rawFields; attempts differ from persuasion success. |
| 3.1 Cross-evaluation experiments | extracted | pp.17–18Exact prose rates retained; unlabeled bars not estimated. Approximate prose category rates not promoted to exact point estimates. |
| 3.2 PsychosisBench | extracted | pp.19–20All 27 labelled DCS/HES/SIS values; source has an SIS direction conflict. |
| 4.1 SAD-mini | extracted | pp.21–22Complete 30-cell table including raw ±95% bootstrap uncertainty and sample sizes; settings not identified. |
| 4.1 GDM Situational Awareness / Stealth | extracted | pp.23–24Exact plotted labels retained; reused comparison bars and original target clearly distinguished. |
| 4.2 Shutdown Resistance | extracted | p.24Complete 16-cell table; no inferred counts from fractional percentages. |
| 4.2 ImpossibleBench | extracted | p.25Complete six-value table; tools conditions preserved. |
| 4.2 AbstentionBench / CharXiv | extracted | pp.25–26Exact 10 chart labels; CharXiv setup comparability caveat retained. |
| 4.3 DeceptionBench | extracted | p.27Exact 12 printed labels rather than rounded prose; explicit inducement is not spontaneous propensity. |
| 4.3 MASK | extracted | pp.27–28All 21 labelled bars; original Sonnet control separated from reused five older controls. |
| 4.3 Alignment Faking | extracted | pp.28–29Compliance gaps and deceptive-alignment rates extracted; awareness confound retained. |
| 4.3 Anthropic Agentic Misalignment | extracted | pp.29–31Panel + three target scenario rates extracted; no harmful action scripts transcribed. |
| 4.3 Machiavelli / TextQuests | extracted | pp.31–32All ten labelled values including CAIS-reported and own target run; unresolved discrepancy flagged. |
| 5.1 IHEval | extracted | pp.33–34Five printed values; plotted setting not disclosed. |
| 5.1 Backbone Breaker | extracted | pp.34–35All ten printed score labels; metadata only, no payload/task instructions. |
| 5.1 AgentHarm | extracted | pp.35–3728 comparative chart values, prose rounding and 16 per-category labels retained; condition ambiguities noted. |
| 5.2.1 FAR.AI | reused | p.37External official-API red-team report reproduced with permission. Mixed CBRN results/numerical dangerous comparisons and operational methods excluded; qualitative safeguard brittleness only. |
| 5.2.2 In-house StrongREJECT | extracted | p.38Complete 16-cell table; all four models tested by Neo; no jailbreak template text. |
| 6.1 Evaluation awareness trend | extracted | pp.39–40Only exact prose endpoint rates retained. Intermediate plotted points and approximate Western line left figure-only. Same-protocol comparator note retained. |
| 6.2 Existing Benchmarks / contamination | extracted | pp.40–41Complete 45-cell awareness table and exact named-dataset probe rates; ranges retained without midpoint; contamination caveat. |
| 6.3 Apollo scenarios | extracted | pp.41–42Safe awareness-only results retained; sandbagging biological-domain numeric comparisons excluded. Novel/published aggregate chart labels retained. |
| 6.4 Awareness case studies / six-eval monitor | extracted | pp.42–44MASK count and all 42 monitor table values; causal-influence attribution only a heuristic; no quoted chain-of-thought. |
| 7 Petri / Bloom | extracted | pp.45–46Audit metadata and complete ablation score/delta table; operational persona/task descriptions omitted. |
| 8.1 JailbreakBench safety judges | extracted | pp.47–48Complete 33-cell validation table, graded n and false-positive/negative metrics; varying judge refusal coverage. |
| 8.2 MASK judges | extracted | pp.48–4918 distribution labels, 12 dissent table cells, complete-case counts; target and judge roles explicit. |
| 8.3 Petri auditor/judge comparisons | extracted | pp.49–53Safe auditor comparison rows, safe selected variance tables, all aggregate bloc statistics. Chemical-weapon/attack scenario rows intentionally excluded; detailed task narratives omitted. |
| Conclusion & Recommendations | qualitative | p.54Scenario-dependent misbehavior, safeguard robustness, awareness and measurement caveats summarized without extending to real-world outcomes. |
| Acknowledgments | qualitative | p.55Yong Zheng-Xin helped run EVMbench; FAR.AI gave reproduction permission; fiscal sponsor Singapore AI Safety Hub. No inferred executor roles from acknowledgments. |
来源待核实事项
不同评测意识监测器的标签与结果不能直接互换，来源没有建立意识对行为的因果影响。
关闭阻碍表的百分比与文字中的样本量无法完整核对，未据此推算成功次数。
FORTRESS 在本报告中仅用于特定子集，位于相关领域评测章节；对抗稳健性章节中的其他评测不能归入 FORTRESS。
版本指纹
document
https://neoresearch.ai/papers/DSv4_Safety_Evaluation_v1.1.pdf
e8d83775830fdd431d6462953a9c11e39a3d6ad25410b47efd600b0ae0ce95d8data
https://neoresearch.ai/research/deepseek-v4-pro-safety-evaluation/
43dd23e7277661bc2c9363bbfff561dcc56dbfbfed45519dadb5fb6e1e71ab33参考文献
- [1] Evaluating DeepSeek v4 Pro for Frontier Risks. 2026-05-29. v1.1 · 29 May 2026