NUWA Risk Governance · Study
Scaling Misuse Detection with Self-Assessment
NUWA research note · independent review pending
NUWA Research Note
Research questionDesign and assumptionsSource findingsNUWA interpretationLimits and contrary evidenceResearch and decision relevanceEvidence updateInstitutions and rolesOriginal resourcesModel-assisted reading · Selected sections of the original full text or official article; a complete section-by-section audit remains pending.
At a glance
Teacher-relative recall, passive monitoring and adaptive blocking are distinct endpoints.
Research question
How do teacher-relative monitoring recall and adaptive attack blocking differ?
Design and assumptions
The source separates production-model offline self-assessment, independently trained anonymous size groups and adaptive blocking on 24 tasks. Offline reference labels come from a teacher classifier. Production classifiers are disabled in the blocking experiment.
Source findings
At a 1% false-positive rate, teacher-relative recall was 78% overall and 94% for high-priority cases. Adaptive attack success was 19/24 without safeguards, 13/24 with self-assessment blocking and 6/24 with a prompted classifier; printed rates were 79%, 54% and 25%.
NUWA interpretation
Teacher-relative recall, traffic coverage and adaptive protection answer different questions. Self-assessment adds observations but retains dependence on label quality and attack distribution. Anonymous XS/S/M/L labels do not disclose parameter counts.
Limits and contrary evidence
Teacher blind spots can propagate. Passive monitoring is not blocking, and the anonymous training experiment is not a matched comparison with the production-model result.
Research and decision relevance
NUWA recommendation: Record the labeling reference, operating point, traffic coverage and adaptive attacker separately before using this evidence in a control argument.
Precise source locations
- Scaling Misuse Detection with Self-Assessment · Results; limitations; text lines 00008,00014,00060Original resource captured 2026-10-28T07:24:41.534870+00:00; sha256:03b9ece6dae1c006622bb47373fb914bec20cee054a452748f30573c63ff6110
Source checking, extraction, editorial approval and replication are recorded separately. Methodology →
Evidence update · 2026-10-28
Model-assisted reading · independent evidence and bilingual editorial review pending. Values are source findings or inventory counts, separate from the trend result series.
Source-bound observations (5)
Counts, rates and contrasts have different units. A sample count is not a number of independent experiments. Unknown numerators and interval levels remain unknown.
| System / metric | Reported value | Population and conditions |
|---|---|---|
| Claude Fable5 Attack success | 79 % | Denominator: 24 offensive-cyber-tasks Numerator: 19 Conditions, limitations and source{
"classifiers": "production classifiers off",
"condition": "No safeguards"
}Adaptive attacker condition; no cross-benchmark ranking. Operational tasks not reproduced in this package. Original source ↗ · Can self-assessment block attacks? Fig.4; text lines 00073Original capture 2026-10-28T07:24:41.534870+00:00; sha256:03b9ece6dae1c006622bb47373fb914bec20cee054a452748f30573c63ff6110 |
| Claude Fable5 Attack success | 54 % | Denominator: 24 offensive-cyber-tasks Numerator: 13 Conditions, limitations and source{
"classifiers": "production classifiers off",
"condition": "Self-assessment blocking"
}Adaptive attacker condition; no cross-benchmark ranking. Operational tasks not reproduced in this package. Original source ↗ · Can self-assessment block attacks? Fig.4; text lines 00073Original capture 2026-10-28T07:24:41.534870+00:00; sha256:03b9ece6dae1c006622bb47373fb914bec20cee054a452748f30573c63ff6110 |
| Claude Fable5 Attack success | 25 % | Denominator: 24 offensive-cyber-tasks Numerator: 6 Conditions, limitations and source{
"classifiers": "production classifiers off",
"condition": "Prompted classifier blocking"
}Adaptive attacker condition; no cross-benchmark ranking. Operational tasks not reproduced in this package. Original source ↗ · Can self-assessment block attacks? Fig.4; text lines 00073Original capture 2026-10-28T07:24:41.534870+00:00; sha256:03b9ece6dae1c006622bb47373fb914bec20cee054a452748f30573c63ff6110 |
| Claude Fable5 Teacher-relative recall | 78 % | Conditions, limitations and source{
"falsePositiveRate": 0.01,
"referenceLabels": "MythosPreview prompted classifier",
"highPriorityThreshold": null
}No sample denominator disclosed here; teacher-relative not independent human-ground-truth recall. Original source ↗ · Results: production Fable5; Fig.1; text lines 00060Original capture 2026-10-28T07:24:41.534870+00:00; sha256:03b9ece6dae1c006622bb47373fb914bec20cee054a452748f30573c63ff6110 |
| Claude Fable5 High-priority teacher-relative recall | 94 % | Conditions, limitations and source{
"falsePositiveRate": 0.01,
"referenceLabels": "MythosPreview prompted classifier",
"highPriorityThreshold": 7
}No sample denominator disclosed here; teacher-relative not independent human-ground-truth recall. Original source ↗ · Results: production Fable5; Fig.1; text lines 00060Original capture 2026-10-28T07:24:41.534870+00:00; sha256:03b9ece6dae1c006622bb47373fb914bec20cee054a452748f30573c63ff6110 |
Related reading
NUWA reading connections; these do not imply citation, replication or score equivalence.
Institutions and roles
Cite and reuse
Cite the original publication for its findings. Cite this NUWA record separately for its explanation and retained snapshot.
Original publication · BibTeX
@misc{self-assessment,
title = {Scaling Misuse Detection with Self-Assessment},
url = {https://safeguards.anthropic.com/2026/self-assessment/},
year = {2026},
date = {2026-10-16},
version = {Original resource captured 2026-10-28T07:24:41.534870+00:00; sha256:03b9ece6dae1c006622bb47373fb914bec20cee054a452748f30573c63ff6110},
note = {Original resource captured 2026-10-28T07:24:41.534870+00:00; sha256:03b9ece6dae1c006622bb47373fb914bec20cee054a452748f30573c63ff6110}
}
NUWA record · BibTeX
@misc{self-assessment-nuwa,
title = {Scaling Misuse Detection with Self-Assessment — NUWA research record},
url = {https://whitzard.tech/en/nuwa/whitzard-index/work/self-assessment/},
version = {2026-10-29.1},
organization = {NUWA Frontier AI Safety Lab},
howpublished = {Retained note edition: https://whitzard.tech/nuwa/whitzard-index/data/record-editions/2026-10-29.1--self-assessment.json},
note = {NUWA research note. Source-reading scope and independent review status are stated on the record. Snapshot 2026-10-29.1. Retained note edition: https://whitzard.tech/nuwa/whitzard-index/data/record-editions/2026-10-29.1--self-assessment.json.}
}
Evidence snapshot 2026-10-29.1 · Index updates · RSS
References
- [1] Scaling Misuse Detection with Self-Assessment. 2026-10-16 ; Results; limitations; text lines 00008,00014,00060