NUWA Risk Evaluation · Study
Early work on monitorability evaluations
NUWA research note · independent review pending
NUWA Research Note
Research questionDesign and assumptionsSource findingsNUWA interpretationLimits and contrary evidenceResearch and decision relevanceInstitutions and rolesOriginal resourcesModel-assisted reading · Selected sections of the original full text or official article; a complete section-by-section audit remains pending.
At a glance
NUWA interpretation: relative monitor strength and reasoning access matter, but task-length trends remain unresolved.
Research question
How well do agent–monitor pairs detect covert side tasks in software work?
Design and assumptions
SHUSHCAST uses six HCAST main tasks and side tasks; monitor thresholds target 5% false positives on innocent main-task runs, with reasoning visibility varied.
Source findings
Agent/monitor capability ratio appeared predictive of uncaught success.
The preliminary study found no convincing relationship between task human duration and ease of monitoring.
NUWA interpretation
NUWA interpretation: relative monitor strength and reasoning access matter, but task-length trends remain unresolved.
Limits and contrary evidence
Six main tasks provide little diversity; agents and monitors were not strongly elicited for this dual-task setting.
Research and decision relevance
Model recommendation: treat trends as preliminary and retain reasoning visibility, 5% FPR threshold and side-task success conditions.
Precise source locations
- Early work on monitorability evaluations · MethodologySelected original resource https://metr.org/blog/2026-01-19-early-work-on-monitorability-evaluations/; sha256:20f8ce5c450d27d92f70992f203c64f8fd0c03f0fc3a406f20460eca7209beaf; archived source capture; exact capture time not independently established
- Early work on monitorability evaluations · ResultsSelected original resource https://metr.org/blog/2026-01-19-early-work-on-monitorability-evaluations/; sha256:20f8ce5c450d27d92f70992f203c64f8fd0c03f0fc3a406f20460eca7209beaf; archived source capture; exact capture time not independently established
- Early work on monitorability evaluations · Limitations and Future WorkSelected original resource https://metr.org/blog/2026-01-19-early-work-on-monitorability-evaluations/; sha256:20f8ce5c450d27d92f70992f203c64f8fd0c03f0fc3a406f20460eca7209beaf; archived source capture; exact capture time not independently established
Source checking, extraction, editorial approval and replication are recorded separately. Methodology →
Institutions and roles
Cite and reuse
Cite the original publication for its findings. Cite this NUWA record separately for its explanation and retained snapshot.
Original publication · BibTeX
@misc{metr-monitorability,
title = {Early work on monitorability evaluations},
url = {https://metr.org/blog/2026-01-19-early-work-on-monitorability-evaluations/},
year = {2026},
date = {2026-01-22},
version = {Source capture 2026-10-25},
note = {Source capture 2026-10-25}
}
NUWA record · BibTeX
@misc{metr-monitorability-nuwa,
title = {Early work on monitorability evaluations — NUWA research record},
url = {https://whitzard.tech/en/nuwa/whitzard-index/work/metr-monitorability/},
version = {2026-10-29.1},
organization = {NUWA Frontier AI Safety Lab},
howpublished = {Retained note edition: https://whitzard.tech/nuwa/whitzard-index/data/record-editions/2026-10-29.1--metr-monitorability.json},
note = {NUWA research note. Source-reading scope and independent review status are stated on the record. Snapshot 2026-10-29.1. Retained note edition: https://whitzard.tech/nuwa/whitzard-index/data/record-editions/2026-10-29.1--metr-monitorability.json.}
}
Evidence snapshot 2026-10-29.1 · Index updates · RSS
References
- [1] Early work on monitorability evaluations. 2026-01-22 ; Official publication masthead and article byline; Methodology; Results