{"schemaVersion":"nuwa-current-record-1","version":"2026-10-29.1","publication":"published","work":{"id":"monitorability-metrics","title":"Beyond g-mean²: complementary metrics for monitorability","aliases":[],"authors":["Melody Guan","Yuxiang Xie","Bowen Baker"],"primaryFunction":"evaluation","secondaryFunctions":[],"kind":"method","sourceId":"monitorability-metrics-publication","attributionSourceId":"monitorability-metrics-publication","originalUrl":"https://alignment.openai.com/beyond-gmean-squared/","publicationDate":"2026-10-20","updatedOn":"2026-10-25","areas":["deception"],"scopeNote":{"en":"Relevance is assigned to the research question and stated conditions of the work. A topic tag is not a risk rating.","zh":"风险关联依据研究问题及明确条件建立，主题标签不代表风险评级。"},"protocolIds":[],"contributions":[{"organizationId":"openai","role":"publisher","sourceId":"monitorability-metrics-publication","locator":"Official publication masthead and article byline"}],"summary":{"en":"Examines complementary monitorability metrics and how evaluation objectives change when a monitor is used in a control protocol.","zh":"研究互补的可监测性指标，以及监测器进入控制协议后评测目标如何变化。"},"note":{"question":null,"method":null,"finding":null,"supports":null,"limits":null,"connections":null},"resources":[],"noteStatus":{"question":"limited","method":"limited","finding":"limited","supports":"limited","limits":"limited","connections":"limited"},"noteAvailability":{"en":"A substantive NUWA research note is not yet available. The repeated draft sections were removed pending review of the original methods, findings and limitations; use the original publication for the research account.","zh":"实质性的 NUWA 研究笔记尚未完成。重复草稿章节已移除，待核查原始方法、发现与限制；研究内容请以原始出版物为准。"},"noteSourceLocator":"No complete section-level method/results/limitations review completed for this note; cached original page and any chart audit retained as discovery evidence.","tags":["risk-monitoring"]},"reading":{"id":"monitorability-metrics","title":"Beyond g-mean²: complementary metrics for monitorability","en":{"question":"How can monitorability comparisons separate instance selection from deployment gating?","design":"The article adds pair-matched comparisons on shared instances and a post-control rejection-gating simulation, plus lower/upper intervention bounds.","findings":["Historical pair-matched rankings generally agreed with dataset-level rankings; disagreements highlighted selection-sensitive or near-tied cases.","Retry policies can restore some risk removed by a monitor gate."],"interpretation":"NUWA interpretation: discrimination, comparable-instance ordering and post-control risk answer different questions.","limits":"Gating estimates depend on retry and deployment assumptions; uncertain causal attribution requires both intervention bounds.","decisionRelevance":"Model recommendation: show deployed-bad and any-deployment rates together rather than optimizing a single monitorability number.","takeaway":"NUWA interpretation: discrimination, comparable-instance ordering and post-control risk answer different questions.","connections":[],"scoring":null,"conditions":null,"statistics":null,"usage":"Model recommendation: show deployed-bad and any-deployment rates together rather than optimizing a single monitorability number."},"zh":{"question":"可监测性比较如何分开样本选择与部署门控？","design":"文章新增共享实例配对比较和拒绝门控模拟，并给干预下界、上界。","findings":["历史配对排名通常与数据集级一致，分歧显示筛选敏感或近似平局。","重试会恢复部分被门控削减的风险。"],"interpretation":"NUWA 解读：判别、可比实例排序与控制后风险回答不同问题。","limits":"门控估计依赖重试与部署假设，因果归因不确定时需同时给上下界。","decisionRelevance":"模型建议：同时展示坏输出部署率及任意部署率，不优化单一分数。","takeaway":"NUWA 解读：判别、可比实例排序与控制后风险回答不同问题。","connections":[],"scoring":null,"conditions":null,"statistics":null,"usage":"模型建议：同时展示坏输出部署率及任意部署率，不优化单一分数。"},"assertionPolicy":{"question":"source-research-question","design":"source-finding","findings":"source-finding","interpretation":"nuwa-interpretation","limits":"source-limit-or-explicit-editorial-scope-limit","connections":"source-supported-relationship-only","decisionRelevance":"model-recommendation-not-source-conclusion"},"sourceRefs":[{"sourceId":"monitorability-metrics-publication","url":"https://alignment.openai.com/beyond-gmean-squared/","locator":"Why one monitorability number is not enough","version":"Selected original resource https://alignment.openai.com/beyond-gmean-squared/; sha256:6c845aa01364667d193dc47aafa490ce24ff4185abb839e9c70109e2258a7ea7; archived source capture; exact capture time not independently established","artifactSha256":"6c845aa01364667d193dc47aafa490ce24ff4185abb839e9c70109e2258a7ea7"},{"sourceId":"monitorability-metrics-publication","url":"https://alignment.openai.com/beyond-gmean-squared/","locator":"Metric 1: compare models on shared instances","version":"Selected original resource https://alignment.openai.com/beyond-gmean-squared/; sha256:6c845aa01364667d193dc47aafa490ce24ff4185abb839e9c70109e2258a7ea7; archived source capture; exact capture time not independently established","artifactSha256":"6c845aa01364667d193dc47aafa490ce24ff4185abb839e9c70109e2258a7ea7"},{"sourceId":"monitorability-metrics-publication","url":"https://alignment.openai.com/beyond-gmean-squared/","locator":"Historical system card results","version":"Selected original resource https://alignment.openai.com/beyond-gmean-squared/; sha256:6c845aa01364667d193dc47aafa490ce24ff4185abb839e9c70109e2258a7ea7; archived source capture; exact capture time not independently established","artifactSha256":"6c845aa01364667d193dc47aafa490ce24ff4185abb839e9c70109e2258a7ea7"},{"sourceId":"monitorability-metrics-publication","url":"https://alignment.openai.com/beyond-gmean-squared/","locator":"Metric 2: simulate the monitor as a rejection gate","version":"Selected original resource https://alignment.openai.com/beyond-gmean-squared/; sha256:6c845aa01364667d193dc47aafa490ce24ff4185abb839e9c70109e2258a7ea7; archived source capture; exact capture time not independently established","artifactSha256":"6c845aa01364667d193dc47aafa490ce24ff4185abb839e9c70109e2258a7ea7"},{"sourceId":"monitorability-metrics-publication","url":"https://alignment.openai.com/beyond-gmean-squared/","locator":"Gating reduces risk; retries give some risk back","version":"Selected original resource https://alignment.openai.com/beyond-gmean-squared/; sha256:6c845aa01364667d193dc47aafa490ce24ff4185abb839e9c70109e2258a7ea7; archived source capture; exact capture time not independently established","artifactSha256":"6c845aa01364667d193dc47aafa490ce24ff4185abb839e9c70109e2258a7ea7"}],"publication":{"state":"published","publishedOn":"2026-10-29","authorizationReference":"user-authorized-global-index-publication-2026-10-29","reviewedCandidateId":"local-6efe31bc4a0c0f37"},"review":{"state":"published-note","sourceScope":"full-text","availableScope":null,"readingCoverage":"selected-source-sections; not a complete section-by-section paper audit","completePaperAudit":false,"independentEvidenceReview":"pending-human-review","bilingualEditorialReview":"pending-human-review","authorship":"model-assisted-research-note","preparedFingerprint":"28bd1e3d7d3a343b7ae33b01ed9eb66fd5667b201f95ca0855cf49cd5d4e07d2"}},"sources":[{"id":"monitorability-metrics-publication","title":"Beyond g-mean²: complementary metrics for monitorability","url":"https://alignment.openai.com/beyond-gmean-squared/","version":"Source capture 2026-10-25","publicationDate":"2026-10-20","datePrecision":"day","type":"research-publication","verifiedOn":"2026-10-25","documentId":"https://alignment.openai.com/beyond-gmean-squared","authors":["Melody Guan","Yuxiang Xie","Bowen Baker"]}]}