{"schemaVersion":"nuwa-current-record-1","version":"2026-10-29.1","publication":"published","work":{"id":"stealth-evals","title":"Evaluating Frontier Models for Stealth and Situational Awareness","aliases":["Stealth evaluations"],"authors":["Mary Phuong","Roland S. Zimmermann","Ziyue Wang","David Lindner","Victoria Krakovna","Sarah Cogan","Allan Dafoe","Lewis Ho","Rohin Shah"],"primaryFunction":"evaluation","secondaryFunctions":[],"kind":"benchmark","sourceId":"stealth-evals","attributionSourceId":"stealth-evals-author-source","originalUrl":"https://arxiv.org/abs/2505.01420v4","publicationDate":"2025-05-02","updatedOn":"2026-10-25","areas":["deception"],"scopeNote":{"en":"Primary dimension follows the source’s measured construct. Metric-level interpretations may differ.","zh":"主维度依据来源实际测量的构念；具体指标可能属于其他维度。"},"protocolIds":["stealth-evals"],"contributions":[{"organizationId":"deepmind","role":"research","sourceId":"stealth-evals-author-source","locator":"Author affiliations on the original paper title page"}],"summary":{"en":"Reasoning prerequisites for evading oversight.","zh":"规避监督所需的推理前置能力。"},"note":{"question":{"en":"Reasoning about oversight and deployment context.","zh":"有关监督机制与部署情境的推理。"},"method":{"en":"Model reasoning about oversight and test context","zh":"推理监督及测试情境的模型"},"finding":{"en":"Reasoning prerequisites for evading oversight.","zh":"规避监督所需的推理前置能力。"},"supports":null,"limits":{"en":"Prerequisite capability is distinct from a tendency to scheme.","zh":"前置能力不同于实施隐蔽谋划的倾向。"},"connections":null},"resources":[],"noteStatus":{"question":"limited","method":"limited","finding":"limited","supports":"limited","limits":"limited","connections":"limited"},"noteAvailability":{"en":"Existing source-specific draft sections are retained with limited review status. A complete methods, findings and limitations check remains pending; generic sections were omitted.","zh":"已有的来源相关草稿章节予以保留，审核状态为有限核查。完整的方法、发现与限制核查仍待完成；通用套话章节已省略。"},"noteSourceLocator":"Retained existing source-specific note sections; this pass audits duplicate/boilerplate structure and source availability, not complete section-level verification.","tags":[]},"reading":{"id":"stealth-evals","title":"Evaluating Frontier Models for Stealth and Situational Awareness","en":{"question":"Can low stealth or situational reasoning support a bounded scheming-inability argument?","design":"The suite uses five stealth and eleven situational-awareness challenges, with challenge-specific pass rules tied to a proposed safety case.","findings":["No evaluated model reached the authors’ concerning capability thresholds.","The proposed case requires passing fewer than 80% in at least one family; failure of the case does not establish scheming capability (§3.3)."],"interpretation":"NUWA interpretation: this is conditional evidence within an explicit safety argument, not a global model safety certificate.","limits":"The authors take no targeted anti-sandbagging measures; capability elicitation and fair human comparisons are load-bearing assumptions.","decisionRelevance":"Model recommendation: examine defeaters and elicitation before relying on low scores as evidence of inability.","takeaway":"NUWA interpretation: this is conditional evidence within an explicit safety argument, not a global model safety certificate.","connections":[],"scoring":"Challenge-specific passes; the proposed safety argument uses 9/11 awareness or 4/5 stealth thresholds.","conditions":"No targeted anti-sandbagging measures; immediate-successor claim is temporally bounded by the paper.","statistics":"The proposed case requires passing fewer than 80% in at least one family; failure of the case does not establish scheming capability (§3.3).","usage":"Model recommendation: examine defeaters and elicitation before relying on low scores as evidence of inability."},"zh":{"question":"低隐蔽能力或情境推理能否支持有边界的谋划无能力论证？","design":"五项隐蔽、十一项情境意识挑战采用分类通过规则，服务于一个明确安全论证。","findings":["受测模型均未达到作者设定的关注阈值。","所提论证要求至少一类通过不足80%；论证不成立不等于已证明能谋划（§3.3）。"],"interpretation":"NUWA 解读：这是安全论证内的条件性证据，不是全局安全认证。","limits":"作者未专门防止能力隐藏；充分激发能力及公平人类比较是关键假设。","decisionRelevance":"模型建议：将低分用于无能力证据前，先审查反驳条件与能力激发。","takeaway":"NUWA 解读：这是安全论证内的条件性证据，不是全局安全认证。","connections":[],"scoring":"各挑战单独通过规则，所提安全论证采用意识9/11或隐蔽4/5阈值。","conditions":"无专门反能力隐藏措施，对直接后继版本论证有时间限制。","statistics":"所提论证要求至少一类通过不足80%；论证不成立不等于已证明能谋划（§3.3）。","usage":"模型建议：将低分用于无能力证据前，先审查反驳条件与能力激发。"},"assertionPolicy":{"question":"source-research-question","design":"source-finding","findings":"source-finding","interpretation":"nuwa-interpretation","limits":"source-limit-or-explicit-editorial-scope-limit","connections":"source-supported-relationship-only","decisionRelevance":"model-recommendation-not-source-conclusion"},"sourceRefs":[{"sourceId":"stealth-evals-author-source","url":"https://arxiv.org/html/2505.01420v4","locator":"Abstract","version":"Selected original resource https://arxiv.org/html/2505.01420v4; sha256:3d6d7f0188fb16f02f90d31a429b996cda133327c9a6f100efd3616f73b19315; archived source capture; exact capture time not independently established","artifactSha256":"3d6d7f0188fb16f02f90d31a429b996cda133327c9a6f100efd3616f73b19315"},{"sourceId":"stealth-evals-author-source","url":"https://arxiv.org/html/2505.01420v4","locator":"§2.3 Possible defeaters","version":"Selected original resource https://arxiv.org/html/2505.01420v4; sha256:3d6d7f0188fb16f02f90d31a429b996cda133327c9a6f100efd3616f73b19315; archived source capture; exact capture time not independently established","artifactSha256":"3d6d7f0188fb16f02f90d31a429b996cda133327c9a6f100efd3616f73b19315"},{"sourceId":"stealth-evals-author-source","url":"https://arxiv.org/html/2505.01420v4","locator":"§3.3 Evaluation thresholds for the safety case","version":"Selected original resource https://arxiv.org/html/2505.01420v4; sha256:3d6d7f0188fb16f02f90d31a429b996cda133327c9a6f100efd3616f73b19315; archived source capture; exact capture time not independently established","artifactSha256":"3d6d7f0188fb16f02f90d31a429b996cda133327c9a6f100efd3616f73b19315"},{"sourceId":"stealth-evals-author-source","url":"https://arxiv.org/html/2505.01420v4","locator":"§7.2 Limitations","version":"Selected original resource https://arxiv.org/html/2505.01420v4; sha256:3d6d7f0188fb16f02f90d31a429b996cda133327c9a6f100efd3616f73b19315; archived source capture; exact capture time not independently established","artifactSha256":"3d6d7f0188fb16f02f90d31a429b996cda133327c9a6f100efd3616f73b19315"}],"publication":{"state":"published","publishedOn":"2026-10-29","authorizationReference":"user-authorized-global-index-publication-2026-10-29","reviewedCandidateId":"local-6efe31bc4a0c0f37"},"review":{"state":"published-note","sourceScope":"full-text","availableScope":null,"readingCoverage":"selected-source-sections; not a complete section-by-section paper audit","completePaperAudit":false,"independentEvidenceReview":"pending-human-review","bilingualEditorialReview":"pending-human-review","authorship":"model-assisted-research-note","preparedFingerprint":"f806d151b7d24a8a24c6c524002b40171780575569e35dbf95c9662500ea80a1"}},"sources":[{"id":"stealth-evals","title":"Evaluating Frontier Models for Stealth and Situational Awareness","url":"https://arxiv.org/abs/2505.01420v4","version":"arXiv:2505.01420v4","publicationDate":"2025-07-03","datePrecision":"day","type":"benchmark","verifiedOn":"2026-10-24","documentId":"arxiv:2505.01420","authors":["Mary Phuong","Roland S. Zimmermann","Ziyue Wang","David Lindner","Victoria Krakovna","Sarah Cogan","Allan Dafoe","Lewis Ho","Rohin Shah"],"aliases":["https://doi.org/10.48550/arXiv.2505.01420","https://arxiv.org/abs/2505.01420","https://arxiv.org/pdf/2505.01420"]}]}