{"schemaVersion":"nuwa-current-record-1","version":"2026-10-29.1","publication":"published","work":{"id":"extreme-risk-evaluations","title":"Model evaluation for extreme risks","aliases":[],"authors":["Toby Shevlane","Sebastian Farquhar","Ben Garfinkel","Mary Phuong","Jess Whittlestone","Jade Leung","Daniel Kokotajlo","Nahema Marchal","Markus Anderljung","Noam Kolt","Lewis Ho","Divya Siddarth","Shahar Avin","Will Hawkins","Been Kim","Iason Gabriel","Vijay Bolina","Jack Clark","Yoshua Bengio","Paul Christiano","Allan Dafoe"],"primaryFunction":"governance","secondaryFunctions":[],"kind":"technical-report","sourceId":"extreme-risk-evaluations-publication","attributionSourceId":"extreme-risk-evaluations-publication-author-source","originalUrl":"https://arxiv.org/abs/2305.15324","publicationDate":"2023-05-24","updatedOn":"2026-10-25","areas":[],"scopeNote":{"en":"Relevance is assigned to the research question and stated conditions of the work. A topic tag is not a risk rating.","zh":"风险关联依据研究问题及明确条件建立，主题标签不代表风险评级。"},"protocolIds":[],"contributions":[{"organizationId":"deepmind","role":"research","sourceId":"extreme-risk-evaluations-publication-author-source","locator":"Author affiliations on the original paper title page"},{"organizationId":"openai","role":"research","sourceId":"extreme-risk-evaluations-publication-author-source","locator":"Author affiliations on the original paper title page"},{"organizationId":"toronto","role":"research","sourceId":"extreme-risk-evaluations-publication-author-source","locator":"Author affiliations on the original paper title page"},{"organizationId":"oxford","role":"research","sourceId":"extreme-risk-evaluations-publication-author-source","locator":"Author affiliations on the original paper title page"},{"organizationId":"anthropic","role":"research","sourceId":"extreme-risk-evaluations-publication-author-source","locator":"Author affiliations on the original paper title page"},{"organizationId":"montreal","role":"research","sourceId":"extreme-risk-evaluations-publication-author-source","locator":"Author affiliations on the original paper title page"},{"organizationId":"mila","role":"research","sourceId":"extreme-risk-evaluations-publication-author-source","locator":"Author affiliations on the original paper title page"}],"summary":{"en":"Develops an approach to evaluating extreme model risks, connecting capability assessment with responsible development decisions.","zh":"提出极端模型风险评测思路，将能力评估与负责任的开发决策联系起来。"},"note":{"question":null,"method":null,"finding":null,"supports":null,"limits":null,"connections":null},"resources":[],"noteStatus":{"question":"limited","method":"limited","finding":"limited","supports":"limited","limits":"limited","connections":"limited"},"noteAvailability":{"en":"A substantive NUWA research note is not yet available. The repeated draft sections were removed pending review of the original methods, findings and limitations; use the original publication for the research account.","zh":"实质性的 NUWA 研究笔记尚未完成。重复草稿章节已移除，待核查原始方法、发现与限制；研究内容请以原始出版物为准。"},"noteSourceLocator":"No complete section-level method/results/limitations review completed for this note; cached original page and any chart audit retained as discovery evidence.","tags":[]},"reading":{"id":"extreme-risk-evaluations","title":"Model evaluation for extreme risks","en":{"takeaway":"The paper places evaluations inside training, deployment, security and external accountability decisions.","question":"How should evaluation evidence constrain frontier development when tests cannot cover every risk?","design":"A conceptual framework separates dangerous capabilities and alignment, combines internal evaluation, external research access and external audit, and links results to governance responses.","findings":["The proposed workflow includes both pre-training assessment and ongoing evaluation during development.","The paper explicitly discusses unknown threats, capability overhang, deceptive behavior and hazards created by evaluation work itself."],"interpretation":"NUWA interpretation: governance value depends on what decisions evidence can change, not only whether a benchmark is technically sophisticated.","limits":"This is a governance blueprint, not a validated risk estimator; factors outside the model and an immature evaluation ecosystem limit assurance.","connections":["NUWA connection: training-run-evaluations and embedded-evaluators elaborate deeper access; misuse-safeguard-principles makes requirement-to-evidence links more concrete."],"decisionRelevance":"NUWA recommendation: attach each evaluation to a decision owner, response rule and residual uncertainty, and assess dissemination hazards separately."},"zh":{"takeaway":"论文将评测纳入训练、部署、安保及外部问责决策。","question":"测试不能覆盖所有风险时，评测证据应如何约束前沿开发？","design":"概念框架区分危险能力与对齐，结合内部评测、外部研究访问及外部审计，再将结果连接治理响应。","findings":["提议流程既包含训练前评估，也包含开发中的持续评测。","论文明确讨论未知威胁、潜在能力、欺骗行为及评测工作自身的信息风险。"],"interpretation":"NUWA 解读：治理价值取决于证据能改变什么决策，而不只取决于基准技术复杂度。","limits":"这是治理蓝图，不是验证过的风险估计器；模型外因素与不成熟的评估生态限制保证力度。","connections":["NUWA 关联：训练过程和嵌入式评估细化深入访问；防护评估原则具体化要求与证据联系。"],"decisionRelevance":"NUWA 建议：每项评测绑定决策负责人、响应规则和残余不确定性，并单独评估传播风险。"},"sourceRefs":[{"sourceId":"extreme-risk-evaluations-publication","url":"https://arxiv.org/html/2305.15324","locator":"2 Extreme risks from general-purpose models","version":"Selected original resource https://arxiv.org/html/2305.15324; sha256:03db3f9243f2a9495a741bb7ed9e332f890302a9abd69b22a6181d2288129639; archived source capture; exact capture time not independently established","artifactSha256":"03db3f9243f2a9495a741bb7ed9e332f890302a9abd69b22a6181d2288129639"},{"sourceId":"extreme-risk-evaluations-publication","url":"https://arxiv.org/html/2305.15324","locator":"3 Model evaluation as critical governance infrastructure","version":"Selected original resource https://arxiv.org/html/2305.15324; sha256:03db3f9243f2a9495a741bb7ed9e332f890302a9abd69b22a6181d2288129639; archived source capture; exact capture time not independently established","artifactSha256":"03db3f9243f2a9495a741bb7ed9e332f890302a9abd69b22a6181d2288129639"},{"sourceId":"extreme-risk-evaluations-publication","url":"https://arxiv.org/html/2305.15324","locator":"4 Building evaluations for extreme risk","version":"Selected original resource https://arxiv.org/html/2305.15324; sha256:03db3f9243f2a9495a741bb7ed9e332f890302a9abd69b22a6181d2288129639; archived source capture; exact capture time not independently established","artifactSha256":"03db3f9243f2a9495a741bb7ed9e332f890302a9abd69b22a6181d2288129639"},{"sourceId":"extreme-risk-evaluations-publication","url":"https://arxiv.org/html/2305.15324","locator":"5 Limitations and hazards","version":"Selected original resource https://arxiv.org/html/2305.15324; sha256:03db3f9243f2a9495a741bb7ed9e332f890302a9abd69b22a6181d2288129639; archived source capture; exact capture time not independently established","artifactSha256":"03db3f9243f2a9495a741bb7ed9e332f890302a9abd69b22a6181d2288129639"}],"publication":{"state":"published","publishedOn":"2026-10-29","authorizationReference":"user-authorized-global-index-publication-2026-10-29","reviewedCandidateId":"local-6efe31bc4a0c0f37"},"review":{"state":"published-note","sourceScope":"full-text","availableScope":null,"readingCoverage":"selected substantive sections of the full source body","completePaperAudit":false,"independentEvidenceReview":"pending-human-review","bilingualEditorialReview":"pending-human-review","authorship":"model-assisted-research-note","preparedFingerprint":"70a6eeaaed1aacd90542308ad80b07facf392444a131349ab64a539a8c01df67"}},"sources":[{"id":"extreme-risk-evaluations-publication","title":"Model evaluation for extreme risks","url":"https://arxiv.org/abs/2305.15324","version":"arXiv:2305.15324v2","publicationDate":"2023-05-24","datePrecision":"day","type":"paper","verifiedOn":"2026-10-25","documentId":"arxiv:2305.15324","authors":["Toby Shevlane","Sebastian Farquhar","Ben Garfinkel","Mary Phuong","Jess Whittlestone","Jade Leung","Daniel Kokotajlo","Nahema Marchal","Markus Anderljung","Noam Kolt","Lewis Ho","Divya Siddarth","Shahar Avin","Will Hawkins","Been Kim","Iason Gabriel","Vijay Bolina","Jack Clark","Yoshua Bengio","Paul Christiano","Allan Dafoe"],"aliases":["https://doi.org/10.48550/arXiv.2305.15324","https://arxiv.org/abs/2305.15324","https://arxiv.org/pdf/2305.15324"]}]}