NUWA DEVELOPER DISCLOSURE · Meta
Muse Spark 1.1 Evaluation Report
Developer documentation covering Muse Spark 1.1.
Published 2026-07-09
Sources ↓Developer-authored disclosure. Selected source sections were read; numerical results and comparability have not been audited. NUWA has not independently replicated these experiments.
Models covered
Cybench
Task completion on 40 professional CTF problems.
A protocol mention does not establish cross-report comparability or independent replication.
This section is catalogued; no precise numerical result has been extracted here. Open the original report for its figures and full findings.
CyberGym
Validated reproduction across a corpus of historical vulnerabilities.
A protocol mention does not establish cross-report comparability or independent replication.
This section is catalogued; no precise numerical result has been extracted here. Open the original report for its figures and full findings.
ExploitGym
Evaluates exploit construction across real software targets and checks whether success uses the designated vulnerability.
A protocol mention does not establish cross-report comparability or independent replication.
This section is catalogued; no precise numerical result has been extracted here. Open the original report for its figures and full findings.
SHADE-Arena
Covert side-task completion and monitor detection under defined tool access.
A protocol mention does not establish cross-report comparability or independent replication.
This section is catalogued; no precise numerical result has been extracted here. Open the original report for its figures and full findings.
MASK
Condition-bounded frequency of lying under stated pressures.
A protocol mention does not establish cross-report comparability or independent replication.
This section is catalogued; no precise numerical result has been extracted here. Open the original report for its figures and full findings.
FORTRESS
Evaluates safeguards on national-security and public-safety tasks, separating hazardous requests from benign-use utility.
A protocol mention does not establish cross-report comparability or independent replication.
This section is catalogued; no precise numerical result has been extracted here. Open the original report for its figures and full findings.
AgentDojo
Targeted attack success and user-task utility in a tool-use environment.
A protocol mention does not establish cross-report comparability or independent replication.
This section is catalogued; no precise numerical result has been extracted here. Open the original report for its figures and full findings.
DeceptionBench
Evaluates 150 authored scenarios across five social domains, including neutral, incentive and pressure conditions.
A protocol mention does not establish cross-report comparability or independent replication.
This section is catalogued; no precise numerical result has been extracted here. Open the original report for its figures and full findings.
Agentic Misalignment
Conditional rates and observed actions in controlled simulations.
A protocol mention does not establish cross-report comparability or independent replication.
This section is catalogued; no precise numerical result has been extracted here. Open the original report for its figures and full findings.
Report coverage
| Section | Source location | Treatment |
|---|---|---|
| 2 Preparedness Assessment | PDF §§2.1–2.3 | Linked protocol API agentic affordances expand the assessment context. Developer cannot rule out high pre-mitigation cyber and biochemical capability; residual-risk judgment includes safeguards. |
| 3 Adversarial Robustness | PDF §3 | Linked protocol Direct misuse, prompt injection and false refusals are evaluated separately. |
| 4 Model Behavior | PDF §4 | Linked protocol Honesty, risk escalation, evaluation awareness and automated alignment audits are distinct behavior constructs; artificial scenario results have limited deployment generality. |
References
- [1] Muse Spark 1.1 Evaluation Report. 2026-07-09