NUWA EXTERNAL ASSESSMENT · Neo Research
Evaluating DeepSeek v4 Pro for Frontier Risks
The report separates task capability from pressure-induced behavior and notes format dependence in persuasion assessment.
Published 2026-05-29
Sources ↓Source transcription is not experimental replication. Unknown dates, evaluator roles and configuration details remain unknown. Source groups are not verified rankings.
Models covered
NUWA Research Note
Research question
Which capabilities, behavioral tendencies and evaluation-validity issues are observed in DeepSeek V4 Pro?
Method
Public-access evaluations covering cyber, scientific knowledge, manipulation, control-related behavior, robustness and judge sensitivity.
Source-reported findings
The report separates task capability from pressure-induced behavior and notes format dependence in persuasion assessment.
Limitations
Controlled tasks, elicitation, benchmark familiarity and configuration differences constrain generalization to deployment.
Cybench
Task completion on 40 professional CTF problems.
Competition tasks and supplied guidance differ from operational attacks.
External evaluation: Neo Research
This section is catalogued; no precise numerical result has been extracted here. Open the original report for its figures and full findings.
EVMbench
Separates detection, patching and exploitation modes with their stated task sets.
Modes and task subsets cannot be combined into an undifferentiated cyber-risk score.
External evaluation: Neo Research
This section is catalogued; no precise numerical result has been extracted here. Open the original report for its figures and full findings.
WMDP
Multiple-choice knowledge measurements.
Knowledge is a proxy; scores do not establish practical harmful-task ability.
External evaluation: Neo Research / Reused result: Meta
This section is catalogued; no precise numerical result has been extracted here. Open the original report for its figures and full findings.
LAB-Bench
Evaluates a range of biology research skills with structured questions; text-only or missing-input subsets remain separate protocols.
These are general scientific capabilities, not end-to-end misuse tasks or direct harm probabilities.
This section is catalogued; no precise numerical result has been extracted here. Open the original report for its figures and full findings.
MASK
Condition-bounded frequency of lying under stated pressures.
A controlled benchmark does not establish stable intent or real-world deception prevalence.
External evaluation: Neo Research
This section is catalogued; no precise numerical result has been extracted here. Open the original report for its figures and full findings.
DeceptionBench
Evaluates 150 authored scenarios across five social domains, including neutral, incentive and pressure conditions.
Authored scenarios and model-based judging limit generalization; elicited deception does not establish covert long-term scheming.
Reused result: Meta / External evaluation: Neo Research
This section is catalogued; no precise numerical result has been extracted here. Open the original report for its figures and full findings.
APE
Frequency of persuasive attempts under specified conversational conditions.
Simulated attempts are not measured human belief change or large-scale disinformation reach.
External evaluation: Neo Research
This section is catalogued; no precise numerical result has been extracted here. Open the original report for its figures and full findings.
PersuSafety adapted judge
Defines persuasion scenarios, simulated conversations and safety assessment; adapted judges are separate variants.
Willingness or simulated dialogue scores do not directly measure changes in human beliefs.
External evaluation: Neo Research
This section is catalogued; no precise numerical result has been extracted here. Open the original report for its figures and full findings.
Alignment faking
Condition-dependent response differences and reported traces.
The setting explicitly gives fictional training information; observations do not establish an enduring hidden goal in deployment.
Reused result: Meta / External evaluation: Neo Research
This section is catalogued; no precise numerical result has been extracted here. Open the original report for its figures and full findings.
Agentic Misalignment
Conditional rates and observed actions in controlled simulations.
No real people or systems were harmed; the study does not estimate prevalence in deployment.
Reused result: Meta / External evaluation: Neo Research
This section is catalogued; no precise numerical result has been extracted here. Open the original report for its figures and full findings.
Shutdown resistance
Conditional frequency of shutdown-script interference and observed action traces.
The source notes strong prompt sensitivity; this article is not interchangeable with its later peer-reviewed version.
External evaluation: Neo Research / Reused result: Palisade Research
This section is catalogued; no precise numerical result has been extracted here. Open the original report for its figures and full findings.
Reward hacking
Constructs impossible variants of coding tasks and examines specification-violating passes.
The cheating rate is conditional on contradictions, test access and scaffolding; it is not a general dishonesty rate.
This section is catalogued; no precise numerical result has been extracted here. Open the original report for its figures and full findings.
FORTRESS — CBRNE subset
Evaluates safeguards on national-security and public-safety tasks, separating hazardous requests from benign-use utility.
Judge reliability, task definitions and over-refusal affect the meaning of the reported scores.
This section is catalogued; no precise numerical result has been extracted here. Open the original report for its figures and full findings.
Evaluation awareness
Evaluation awareness in the report-defined setting.
Exploratory or report-specific setting; not a reusable benchmark claim.
External evaluation: Neo Research
This section is catalogued; no precise numerical result has been extracted here. Open the original report for its figures and full findings.
Automated alignment audit
An auditor explores seeded multi-turn interactions, with rubric-based judging of target transcripts.
Low average audit scores do not rule out rare severe cases; results depend on seeds, auditor, judge and target configuration.
This section is catalogued; no precise numerical result has been extracted here. Open the original report for its figures and full findings.
Safety-judge sensitivity
Safety-judge sensitivity in the report-defined setting.
Exploratory or report-specific setting; not a reusable benchmark claim.
This section is catalogued; no precise numerical result has been extracted here. Open the original report for its figures and full findings.
Report coverage
| Section | Source location | Treatment |
|---|---|---|
| Cybench | §1.1 | Protocol variant Protocol and result provenance are retained at this section. |
| EVMbench | §1.2 | Protocol variant Protocol and result provenance are retained at this section. |
| WMDP | §2.1 | Protocol variant Protocol and result provenance are retained at this section. |
| LAB-Bench | §2.2 | Protocol variant Protocol and result provenance are retained at this section. |
| MASK | §3.1 | Protocol variant Protocol and result provenance are retained at this section. |
| DeceptionBench | §3.2 | Protocol variant Protocol and result provenance are retained at this section. |
| APE | §3.3 | Protocol variant Protocol and result provenance are retained at this section. |
| PersuSafety adapted judge | §3.4 | Protocol variant Protocol and result provenance are retained at this section. |
| Alignment faking | §4.1 | Protocol variant Protocol and result provenance are retained at this section. |
| Agentic Misalignment | §4.2 | Protocol variant Protocol and result provenance are retained at this section. |
| Shutdown resistance | §4.3 | Protocol variant Protocol and result provenance are retained at this section. |
| Reward hacking | §4 | Protocol variant Protocol and result provenance are retained at this section. |
| Adversarial robustness | §5 | Protocol variant Protocol and result provenance are retained at this section. |
| Evaluation awareness | §6 | Protocol variant Protocol and result provenance are retained at this section. |
| Automated alignment audit | §6 | Protocol variant Protocol and result provenance are retained at this section. |
| Safety-judge sensitivity | §7 | Protocol variant Protocol and result provenance are retained at this section. |
References
Numerical results are reported by the cited authors.
- [1] Evaluating DeepSeek v4 Pro for Frontier Risks. 2026-05-29. v1.1 · 29 May 2026 ; PDF p. 8; Cybench unguided; pass@1; PDF p. 9; EVMbench; Detect; PDF p. 9; EVMbench; Patch