{"schemaVersion":"nuwa-current-record-1","version":"2026-10-29.1","publication":"published","work":{"id":"monitoring-monitorability","title":"Monitoring Monitorability","aliases":[],"authors":["Melody Y. Guan","Miles Wang","Micah Carroll","Zehao Dou","Annie Y. Wei","Marcus Williams","Benjamin Arnav","Joost Huizinga","Ian Kivlichan","Mia Glaese","Jakub Pachocki","Bowen Baker"],"primaryFunction":"evaluation","secondaryFunctions":[],"kind":"method","sourceId":"monitoring-monitorability-publication","attributionSourceId":"monitoring-monitorability-publication-author-source","originalUrl":"https://arxiv.org/abs/2512.18311","publicationDate":"2025-12-20","updatedOn":"2026-10-25","areas":["deception"],"scopeNote":{"en":"Relevance is assigned to the research question and stated conditions of the work. A topic tag is not a risk rating.","zh":"风险关联依据研究问题及明确条件建立，主题标签不代表风险评级。"},"protocolIds":[],"contributions":[{"organizationId":"openai","role":"research","sourceId":"monitoring-monitorability-publication-author-source","locator":"Paper author block and correspondence at openai.com"}],"summary":{"en":"Defines monitorability evaluations and studies how reasoning access, inference compute and follow-up questions affect detection.","zh":"定义可监测性评测，研究推理可见性、推理计算与追问如何影响检测。"},"note":{"question":{"en":"How reliably can a monitor identify safety-relevant properties from an agent’s reasoning?","zh":"监测器能否可靠地从智能体推理中识别安全相关特征？"},"method":{"en":"The study defines evaluation archetypes and a discrimination metric, and varies reasoning visibility, compute and follow-up questions.","zh":"研究定义评测类型与区分指标，并改变推理可见性、计算和追问条件。"},"finding":{"en":"The authors find benefits from reasoning access in tested settings, with important variation across agent and monitor configurations.","zh":"作者发现推理可见性在受测条件下有助于监测，但效果随智能体和监测器配置显著变化。"},"supports":{"en":"Supports protocol-specific monitor evaluation and explicit treatment of observation access and compute.","zh":"支持按协议评测监测器，并明确观测权限与计算投入。"},"limits":{"en":"Monitorability depends on the agent–monitor pair and evaluation distribution; it is not a measure of overall model safety.","zh":"可监测性依赖智能体与监测器组合和评测分布，不是模型整体安全指标。"},"connections":null},"resources":[],"noteStatus":{"question":"limited","method":"limited","finding":"limited","supports":"limited","limits":"limited","connections":"limited"},"noteAvailability":{"en":"Existing source-specific draft sections are retained with limited review status. A complete methods, findings and limitations check remains pending; generic sections were omitted.","zh":"已有的来源相关草稿章节予以保留，审核状态为有限核查。完整的方法、发现与限制核查仍待完成；通用套话章节已省略。"},"noteSourceLocator":"Retained existing source-specific note sections; this pass audits duplicate/boilerplate structure and source availability, not complete section-level verification.","tags":["risk-monitoring"]},"reading":{"id":"monitoring-monitorability","title":"Monitoring Monitorability","en":{"question":"How can monitorability be measured across different kinds of observable behavior?","design":"The paper defines intervention, process and outcome-property evaluations, multiple monitor scopes and dataset-level g-mean² metrics.","findings":["CoT monitoring outperformed action-only monitoring in practical tested settings.","Longer reasoning was generally more monitorable; studied frontier-scale RL did not materially reduce monitorability."],"interpretation":"NUWA interpretation: monitorability is a two-sided agent–monitor measurement, conditional on evaluation archetype and scope.","limits":"Intervention estimates can be undefined or unstable when effects are nonpositive or tiny; present training-scale results do not guarantee future preservation.","decisionRelevance":"Model recommendation: cite archetype, scope, inclusion criteria and uncertainty with any aggregate monitorability score.","takeaway":"NUWA interpretation: monitorability is a two-sided agent–monitor measurement, conditional on evaluation archetype and scope.","connections":[],"scoring":"Archetype-specific g-mean² monitorability with explicit visibility scopes.","conditions":null,"statistics":null,"usage":"Model recommendation: cite archetype, scope, inclusion criteria and uncertainty with any aggregate monitorability score."},"zh":{"question":"如何跨行为类型测量可监测性？","design":"论文定义干预、过程、结果属性三类评测，多种监测范围及数据集级g-mean²指标。","findings":["受测现实式设置中思维链监测优于仅行动。","更长推理通常更可监测，所测前沿规模RL未明显降低可监测性。"],"interpretation":"NUWA 解读：可监测性是双方测量，依赖评测类型和可见范围。","limits":"干预效应非正或很小时估计可不定义或不稳定；当前规模不能保证未来保持。","decisionRelevance":"模型建议：综合分旁引用类型、范围、纳入标准及不确定性。","takeaway":"NUWA 解读：可监测性是双方测量，依赖评测类型和可见范围。","connections":[],"scoring":"按评测类型定义g-mean²，并明确可见范围。","conditions":null,"statistics":null,"usage":"模型建议：综合分旁引用类型、范围、纳入标准及不确定性。"},"assertionPolicy":{"question":"source-research-question","design":"source-finding","findings":"source-finding","interpretation":"nuwa-interpretation","limits":"source-limit-or-explicit-editorial-scope-limit","connections":"source-supported-relationship-only","decisionRelevance":"model-recommendation-not-source-conclusion"},"sourceRefs":[{"sourceId":"monitoring-monitorability-publication-author-source","url":"https://arxiv.org/html/2512.18311","locator":"Abstract","version":"Selected original resource https://arxiv.org/html/2512.18311; sha256:3c73b648d18ee87eca3d3b306dedc3746665058c627189715b0662e4985d31bd; archived source capture; exact capture time not independently established","artifactSha256":"3c73b648d18ee87eca3d3b306dedc3746665058c627189715b0662e4985d31bd"},{"sourceId":"monitoring-monitorability-publication-author-source","url":"https://arxiv.org/html/2512.18311","locator":"§2 Evaluating Monitorability","version":"Selected original resource https://arxiv.org/html/2512.18311; sha256:3c73b648d18ee87eca3d3b306dedc3746665058c627189715b0662e4985d31bd; archived source capture; exact capture time not independently established","artifactSha256":"3c73b648d18ee87eca3d3b306dedc3746665058c627189715b0662e4985d31bd"},{"sourceId":"monitoring-monitorability-publication-author-source","url":"https://arxiv.org/html/2512.18311","locator":"§3.1.3 Evaluation Design Guidance","version":"Selected original resource https://arxiv.org/html/2512.18311; sha256:3c73b648d18ee87eca3d3b306dedc3746665058c627189715b0662e4985d31bd; archived source capture; exact capture time not independently established","artifactSha256":"3c73b648d18ee87eca3d3b306dedc3746665058c627189715b0662e4985d31bd"},{"sourceId":"monitoring-monitorability-publication-author-source","url":"https://arxiv.org/html/2512.18311","locator":"§9 Discussion and Conclusion","version":"Selected original resource https://arxiv.org/html/2512.18311; sha256:3c73b648d18ee87eca3d3b306dedc3746665058c627189715b0662e4985d31bd; archived source capture; exact capture time not independently established","artifactSha256":"3c73b648d18ee87eca3d3b306dedc3746665058c627189715b0662e4985d31bd"}],"publication":{"state":"published","publishedOn":"2026-10-29","authorizationReference":"user-authorized-global-index-publication-2026-10-29","reviewedCandidateId":"local-6efe31bc4a0c0f37"},"review":{"state":"published-note","sourceScope":"full-text","availableScope":null,"readingCoverage":"selected-source-sections; not a complete section-by-section paper audit","completePaperAudit":false,"independentEvidenceReview":"pending-human-review","bilingualEditorialReview":"pending-human-review","authorship":"model-assisted-research-note","preparedFingerprint":"11be64b27609372f4acd6109499d2aefb0572e63b4419ead340745ed0ad54a65"}},"sources":[{"id":"monitoring-monitorability-publication","title":"Monitoring Monitorability","url":"https://arxiv.org/abs/2512.18311","version":"arXiv:2512.18311v1","publicationDate":"2025-12-20","datePrecision":"day","type":"paper","verifiedOn":"2026-10-25","documentId":"arxiv:2512.18311","authors":["Melody Y. Guan 2 2 2 † \\dagger Equal contribution. Correspondence to Bowen Baker at bowen@openai.com","Miles Wang 2 2 footnotemark: 2","Micah Carroll 2 2 footnotemark: 2","Annie Y. Wei","Marcus Williams","Benjamin Arnav","Ian Kivlichan","Mia Glaese","Jakub Pachocki"],"aliases":["https://doi.org/10.48550/arXiv.2512.18311","https://arxiv.org/abs/2512.18311","https://arxiv.org/pdf/2512.18311"]}]}