{"schemaVersion":"nuwa-current-record-1","version":"2026-10-29.1","publication":"published","work":{"id":"monitorability-opportunity","title":"Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safety","aliases":[],"authors":["Tomek Korbak","Mikita Balesni","Elizabeth Barnes","Yoshua Bengio","Joe Benton","Joseph Bloom","Mark Chen","Alan Cooney","Allan Dafoe","Anca Dragan","Scott Emmons","Owain Evans","David Farhi","Ryan Greenblatt","Dan Hendrycks","Marius Hobbhahn","Evan Hubinger","Geoffrey Irving","Erik Jenner","Daniel Kokotajlo","Victoria Krakovna","Shane Legg","David Lindner","David Luan","Aleksander Mądry","Julian Michael","Neel Nanda","Dave Orr","Jakub Pachocki","Ethan Perez","Mary Phuong","Fabien Roger","Joshua Saxe","Buck Shlegeris","Martín Soto","Eric Steinberger","Jasmine Wang","Wojciech Zaremba","Bowen Baker","Rohin Shah","Vlad Mikulik"],"primaryFunction":"evaluation","secondaryFunctions":[],"kind":"technical-report","sourceId":"monitorability-opportunity-publication","attributionSourceId":"monitorability-opportunity-publication-author-source","originalUrl":"https://arxiv.org/abs/2507.11473","publicationDate":"2025-07-15","updatedOn":"2026-10-25","areas":[],"scopeNote":{"en":"Relevance is assigned to the research question and stated conditions of the work. A topic tag is not a risk rating.","zh":"风险关联依据研究问题及明确条件建立，主题标签不代表风险评级。"},"protocolIds":[],"contributions":[{"organizationId":"aisi","role":"research","sourceId":"monitorability-opportunity-publication-author-source","locator":"Author affiliations on the original paper title page"},{"organizationId":"apollo","role":"research","sourceId":"monitorability-opportunity-publication-author-source","locator":"Author affiliations on the original paper title page"},{"organizationId":"metr","role":"research","sourceId":"monitorability-opportunity-publication-author-source","locator":"Author affiliations on the original paper title page"},{"organizationId":"montreal","role":"research","sourceId":"monitorability-opportunity-publication-author-source","locator":"Author affiliations on the original paper title page"},{"organizationId":"mila","role":"research","sourceId":"monitorability-opportunity-publication-author-source","locator":"Author affiliations on the original paper title page"},{"organizationId":"anthropic","role":"research","sourceId":"monitorability-opportunity-publication-author-source","locator":"Author affiliations on the original paper title page"},{"organizationId":"openai","role":"research","sourceId":"monitorability-opportunity-publication-author-source","locator":"Author affiliations on the original paper title page"},{"organizationId":"deepmind","role":"research","sourceId":"monitorability-opportunity-publication-author-source","locator":"Author affiliations on the original paper title page"},{"organizationId":"truthful","role":"research","sourceId":"monitorability-opportunity-publication-author-source","locator":"Author affiliations on the original paper title page"},{"organizationId":"berkeley","role":"research","sourceId":"monitorability-opportunity-publication-author-source","locator":"Author affiliations on the original paper title page"},{"organizationId":"redwood","role":"research","sourceId":"monitorability-opportunity-publication-author-source","locator":"Author affiliations on the original paper title page"},{"organizationId":"cais","role":"research","sourceId":"monitorability-opportunity-publication-author-source","locator":"Author affiliations on the original paper title page"},{"organizationId":"scale","role":"research","sourceId":"monitorability-opportunity-publication-author-source","locator":"Author affiliations on the original paper title page"}],"summary":{"en":"Identifies chain-of-thought monitoring as a safety research opportunity and analyzes conditions under which it may become less reliable.","zh":"将推理链监测作为安全研究机会，并分析其可靠性可能降低的条件。"},"note":{"question":{"en":"Why might visible chain of thought provide a safety signal, and what could make that opportunity disappear?","zh":"可见思维链为何可能提供安全信号，又有哪些因素会使这一机会消失？"},"method":{"en":"This perspective combines architectural reasoning about serial computation with a review of empirical monitoring evidence. It distinguishes tasks that require externalized reasoning from cases where models merely tend to verbalize it.","zh":"这篇观点文章结合对串行计算的架构分析与经验监测研究综述，区分必须外显推理的任务，以及模型只是倾向于说出推理的情况。"},"finding":{"en":"The authors argue that chain of thought can expose information absent from final outputs without being a complete account of computation. They identify optimization pressures, reasoning in activations and deliberate obfuscation as threats to continued monitorability.","zh":"作者认为，思维链即使不是计算过程的完整记录，也可能暴露最终输出中缺少的信息；优化压力、激活中的推理及主动混淆则可能威胁持续可监测性。"},"supports":{"en":"The work motivates tracking and stress-testing monitorability as an additional safety layer, with separate attention to necessity and propensity to verbalize reasoning.","zh":"该工作支持将可监测性作为额外安全层进行跟踪与压力测试，并分别关注说出推理的必要性与倾向。"},"limits":{"en":"This is a research argument, not a new benchmark or a guarantee. Benign-looking reasoning cannot by itself show alignment, and preserving current chain-of-thought behavior may trade off against other useful safety interventions.","zh":"这是一项研究论证，并非新基准或安全保证。看似正常的推理本身不能证明对齐；保留当前思维链行为还可能与其他有益安全干预存在权衡。"},"connections":null},"resources":[],"noteStatus":{"question":"verified","method":"verified","finding":"verified","supports":"verified","limits":"verified","connections":"limited"},"noteAvailability":{"en":"The substantive sections shown here were checked against the original source. Unfilled sections are omitted; source checking is not experimental replication.","zh":"此处展示的实质性章节已对照原始来源核查；未完成的章节已省略。来源核查不等于实验复现。"},"noteSourceLocator":"arXiv:2507.11473v2, §§1.1–1.2, 2–5","tags":["risk-monitoring"]},"reading":{"id":"monitorability-opportunity","title":"Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safety","en":{"question":"Why preserve natural-language reasoning as an oversight signal?","design":"This research perspective synthesizes architectural arguments and existing monitoring studies, then proposes evaluation and development priorities.","findings":["The authors argue CoT can reveal misbehavior and evaluation failures.","They identify training pressure, deliberate obfuscation and architectural change as possible threats to visibility."],"interpretation":"NUWA interpretation: the source motivates a research agenda; it supplies no new universal monitor success rate.","limits":"CoT monitoring is imperfect and particularly weak evidence where tasks need no verbal reasoning; the argument is not a safety proof.","decisionRelevance":"Model recommendation: track monitorability during development and combine it with independently justified oversight layers.","takeaway":"NUWA interpretation: the source motivates a research agenda; it supplies no new universal monitor success rate.","connections":[],"scoring":null,"conditions":null,"statistics":null,"usage":"Model recommendation: track monitorability during development and combine it with independently justified oversight layers."},"zh":{"question":"为何要保留自然语言推理作为监督信号？","design":"这篇观点论文综合架构论证和既有研究，提出评测与研发方向。","findings":["作者认为思维链可揭示不良行为和评测故障。","训练压力、主动混淆和架构变化可能威胁可见性。"],"interpretation":"NUWA 解读：来源提出研究议程，没有新的通用监测成功率。","limits":"监测并不完美，无需语言推理的任务尤其不能靠它强证安全；论证不是证明。","decisionRelevance":"模型建议：研发中追踪可监测性，并结合另有依据的监督层。","takeaway":"NUWA 解读：来源提出研究议程，没有新的通用监测成功率。","connections":[],"scoring":null,"conditions":null,"statistics":null,"usage":"模型建议：研发中追踪可监测性，并结合另有依据的监督层。"},"assertionPolicy":{"question":"source-research-question","design":"source-finding","findings":"source-finding","interpretation":"nuwa-interpretation","limits":"source-limit-or-explicit-editorial-scope-limit","connections":"source-supported-relationship-only","decisionRelevance":"model-recommendation-not-source-conclusion"},"sourceRefs":[{"sourceId":"monitorability-opportunity-publication-author-source","url":"https://arxiv.org/html/2507.11473","locator":"Abstract","version":"Selected original resource https://arxiv.org/html/2507.11473; sha256:a3aed294c1d4672abe69eaa9c2a3abf1432bf50b3b4869fe6fff8c7eb680ff5f; archived source capture; exact capture time not independently established","artifactSha256":"a3aed294c1d4672abe69eaa9c2a3abf1432bf50b3b4869fe6fff8c7eb680ff5f"},{"sourceId":"monitorability-opportunity-publication-author-source","url":"https://arxiv.org/html/2507.11473","locator":"§1 Chain of Thought Offers a Unique Safety Opportunity","version":"Selected original resource https://arxiv.org/html/2507.11473; sha256:a3aed294c1d4672abe69eaa9c2a3abf1432bf50b3b4869fe6fff8c7eb680ff5f; archived source capture; exact capture time not independently established","artifactSha256":"a3aed294c1d4672abe69eaa9c2a3abf1432bf50b3b4869fe6fff8c7eb680ff5f"},{"sourceId":"monitorability-opportunity-publication-author-source","url":"https://arxiv.org/html/2507.11473","locator":"§2 Chain of Thought Monitorability May be Fragile","version":"Selected original resource https://arxiv.org/html/2507.11473; sha256:a3aed294c1d4672abe69eaa9c2a3abf1432bf50b3b4869fe6fff8c7eb680ff5f; archived source capture; exact capture time not independently established","artifactSha256":"a3aed294c1d4672abe69eaa9c2a3abf1432bf50b3b4869fe6fff8c7eb680ff5f"},{"sourceId":"monitorability-opportunity-publication-author-source","url":"https://arxiv.org/html/2507.11473","locator":"§5 Limitations","version":"Selected original resource https://arxiv.org/html/2507.11473; sha256:a3aed294c1d4672abe69eaa9c2a3abf1432bf50b3b4869fe6fff8c7eb680ff5f; archived source capture; exact capture time not independently established","artifactSha256":"a3aed294c1d4672abe69eaa9c2a3abf1432bf50b3b4869fe6fff8c7eb680ff5f"},{"sourceId":"monitorability-opportunity-publication-author-source","url":"https://arxiv.org/html/2507.11473","locator":"§6 Conclusion","version":"Selected original resource https://arxiv.org/html/2507.11473; sha256:a3aed294c1d4672abe69eaa9c2a3abf1432bf50b3b4869fe6fff8c7eb680ff5f; archived source capture; exact capture time not independently established","artifactSha256":"a3aed294c1d4672abe69eaa9c2a3abf1432bf50b3b4869fe6fff8c7eb680ff5f"}],"publication":{"state":"published","publishedOn":"2026-10-29","authorizationReference":"user-authorized-global-index-publication-2026-10-29","reviewedCandidateId":"local-6efe31bc4a0c0f37"},"review":{"state":"published-note","sourceScope":"full-text","availableScope":null,"readingCoverage":"selected-source-sections; not a complete section-by-section paper audit","completePaperAudit":false,"independentEvidenceReview":"pending-human-review","bilingualEditorialReview":"pending-human-review","authorship":"model-assisted-research-note","preparedFingerprint":"a3022f7e4ea507dd1f1a351e09074967411bfa6bf31f2c4a3c39b75d215580ed"}},"sources":[{"id":"monitorability-opportunity-publication","title":"Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safety","url":"https://arxiv.org/abs/2507.11473","version":"arXiv:2507.11473v2","publicationDate":"2025-07-15","datePrecision":"day","type":"paper","verifiedOn":"2026-10-25","documentId":"arxiv:2507.11473","authors":["Tomek Korbak * UK AI Security Institute Mikita Balesni * Apollo Research Elizabeth Barnes METR Yoshua Bengio University of Montreal & Mila Joe Benton Anthropic Joseph Bloom UK AI Security Institute Mark Chen OpenAI Alan Cooney UK AI Security Institute Allan Dafoe Google DeepMind Anca Dragan Google DeepMind Scott Emmons Google DeepMind Owain Evans Truthful AI & UC Berkeley David Farhi OpenAI Ryan Greenblatt Redwood Research Dan Hendrycks Center for AI Safety Marius Hobbhahn Apollo Research Evan Hubinger Anthropic Geoffrey Irving UK AI Security Institute Erik Jenner Google DeepMind Daniel Kokotajlo AI Futures Project Victoria Krakovna Google DeepMind Shane Legg Google DeepMind David Lindner Google DeepMind David Luan Amazon Aleksander Mądry OpenAI Julian Michael Scale AI Neel Nanda Google DeepMind Dave Orr Google DeepMind Jakub Pachocki OpenAI Ethan Perez Anthropic Mary Phuong Google DeepMind Fabien Roger Anthropic Joshua Saxe Meta Buck Shlegeris Redwood Research Martín Soto UK AI Security Institute Eric Steinberger Magic Jasmine Wang UK AI Security Institute Wojciech Zaremba OpenAI Bowen Baker † OpenAI Rohin Shah † Google DeepMind Vlad Mikulik † Anthropic"],"aliases":["https://doi.org/10.48550/arXiv.2507.11473","https://arxiv.org/abs/2507.11473","https://arxiv.org/pdf/2507.11473"]}]}