NUWA NUWA analysis · model-assisted · independent review pending
From visible reasoning to a defensible safety argument
Monitoring produces evidence; a control protocol acts on it; a safety case explains why that protocol and its assumptions support a defined decision. Success at one layer does not establish the next.
Evidence snapshot 2026-10-28.2 · 2026-10-27
Evidence and argument
MonitorBench tests whether known decision-critical factors are visible under specified message access and stress conditions. AI Control tests defenses against adaptive intentional subversion and evaluates the safety/usefulness tradeoff of the complete protocol. Safety Cases then asks for an explicit argument connecting system-specific evidence and assumptions to an operating-context claim. The Misuse Safety Case illustrates one such chain: red-team effort curves, a risk model, continuing evaluation and response commitments. These sources do not establish that a high visibility score certifies deployment safety, or that a control result covers external misuse pathways.
Visibility is conditional evidence
MonitorBench separates verification, intervention attribution and monitor flags; concealment effects vary by task and message scope.
MonitorBench: A Comprehensive Benchmark for Chain-of-Thought Monitorability in Large Language Models · §§3.1–3.3,4.1–4.3
The complete defense determines the outcome
AI Control combines monitoring with audits, routing and fallback; naive untrusted monitoring can fail even when a monitor has the ability to recognize a problem.
AI Control: Improving Safety Despite Intentional Subversion · §§2,3.2–3.6,5.1
A safety argument must expose its bridge assumptions
Safety-case methods require a context-specific top claim and linked subclaims; the worked misuse case additionally relies on actor modeling and response commitments. Those are not observations supplied by a monitor score.
Safety Cases: A scalable approach to Frontier AI safety · §§2.1–2.4,3.1–3.2,4; PDF pp.2–14
An example safety case for safeguards against misuse · §§2.4–2.7,3–4
Limits and remaining assumptions
The monitoring tasks have known constructed factors; the control game has unusually reliable audits and a narrow threat model; the safety-case examples are methodological or hypothetical. Their uncertainties accumulate unless the linking assumptions are separately justified. No joint deployment experiment or empirical certification is claimed.
Next research question
Model recommendation: specify the forbidden outcome, trace which monitor signals trigger which intervention, test audit and response failures, and attach each assumption and defeater to its source version. Keep the deployment decision assigned to an authorized human decision-maker.
References
- [1] Han Wang, Yifan Sun, et al.. MonitorBench: A Comprehensive Benchmark for Chain-of-Thought Monitorability in Large Language Models. v2 ; §§3.1–3.3,4.1–4.3
- [2] Ryan Greenblatt ∗ Buck Shlegeris Kshitij Sachan Fabien Roger Redwood Research. AI Control: Improving Safety Despite Intentional Subversion ; §§2,3.2–3.6,5.1
- [3] Safety Cases: A scalable approach to Frontier AI safety. 2025-02-10 ; §§2.1–2.4,3.1–3.2,4; PDF pp.2–14
- [4] An example safety case for safeguards against misuse. 2025-06-05 ; §§2.4–2.7,3–4