Risk Governance · Technical report

Model evaluation for extreme risks

NUWA research note · independent review pending

First published: 2023-05-24Source checked: 2026-10-25

NUWA Research Note

Research question Design and assumptions Source findings NUWA interpretation Limits and contrary evidence Research connections Research and decision relevance Institutions and roles Original resources

Model-assisted reading · Selected sections of the original full text or official article; a complete section-by-section audit remains pending.

At a glance

The paper places evaluations inside training, deployment, security and external accountability decisions.

Research question

How should evaluation evidence constrain frontier development when tests cannot cover every risk?

Design and assumptions

A conceptual framework separates dangerous capabilities and alignment, combines internal evaluation, external research access and external audit, and links results to governance responses.

Source findings

The proposed workflow includes both pre-training assessment and ongoing evaluation during development.

The paper explicitly discusses unknown threats, capability overhang, deceptive behavior and hazards created by evaluation work itself.

NUWA interpretation

NUWA interpretation: governance value depends on what decisions evidence can change, not only whether a benchmark is technically sophisticated.

Limits and contrary evidence

This is a governance blueprint, not a validated risk estimator; factors outside the model and an immature evaluation ecosystem limit assurance.

Research connections

NUWA connection: training-run-evaluations and embedded-evaluators elaborate deeper access; misuse-safeguard-principles makes requirement-to-evidence links more concrete.

Research and decision relevance

NUWA recommendation: attach each evaluation to a decision owner, response rule and residual uncertainty, and assess dissemination hazards separately.

Precise source locations

Model evaluation for extreme risks · 2 Extreme risks from general-purpose modelsSelected original resource https://arxiv.org/html/2305.15324; sha256:03db3f9243f2a9495a741bb7ed9e332f890302a9abd69b22a6181d2288129639; archived source capture; exact capture time not independently established
Model evaluation for extreme risks · 3 Model evaluation as critical governance infrastructureSelected original resource https://arxiv.org/html/2305.15324; sha256:03db3f9243f2a9495a741bb7ed9e332f890302a9abd69b22a6181d2288129639; archived source capture; exact capture time not independently established
Model evaluation for extreme risks · 4 Building evaluations for extreme riskSelected original resource https://arxiv.org/html/2305.15324; sha256:03db3f9243f2a9495a741bb7ed9e332f890302a9abd69b22a6181d2288129639; archived source capture; exact capture time not independently established
Model evaluation for extreme risks · 5 Limitations and hazardsSelected original resource https://arxiv.org/html/2305.15324; sha256:03db3f9243f2a9495a741bb7ed9e332f890302a9abd69b22a6181d2288129639; archived source capture; exact capture time not independently established

Source checking, extraction, editorial approval and replication are recorded separately. Methodology →

Institutions and roles

Research by Google DeepMind OpenAI University of Toronto University of Oxford Anthropic Université de Montréal Mila

Authors (21)

Toby Shevlane · Sebastian Farquhar · Ben Garfinkel · Mary Phuong · Jess Whittlestone · Jade Leung · Daniel Kokotajlo · Nahema Marchal · Markus Anderljung · Noam Kolt · Lewis Ho · Divya Siddarth · Shahar Avin · Will Hawkins · Been Kim · Iason Gabriel · Vijay Bolina · Jack Clark · Yoshua Bengio · Paul Christiano · Allan Dafoe

Cite and reuse

Cite the original publication for its findings. Cite this NUWA record separately for its explanation and retained snapshot.

BibTeX ↓RIS ↓Record + source metadata · JSON ↓Reading coverage · JSON Suggest a correction

Original publication · BibTeX

@misc{extreme-risk-evaluations,
  title = {Model evaluation for extreme risks},
  url = {https://arxiv.org/abs/2305.15324},
  author = {Toby Shevlane and Sebastian Farquhar and Ben Garfinkel and Mary Phuong and Jess Whittlestone and Jade Leung and Daniel Kokotajlo and Nahema Marchal and Markus Anderljung and Noam Kolt and Lewis Ho and Divya Siddarth and Shahar Avin and Will Hawkins and Been Kim and Iason Gabriel and Vijay Bolina and Jack Clark and Yoshua Bengio and Paul Christiano and Allan Dafoe},
  year = {2023},
  date = {2023-05-24},
  version = {arXiv:2305.15324v2},
  note = {arXiv:2305.15324v2}
}

NUWA record · BibTeX

@misc{extreme-risk-evaluations-nuwa,
  title = {Model evaluation for extreme risks — NUWA research record},
  url = {https://whitzard.tech/en/nuwa/whitzard-index/work/extreme-risk-evaluations/},
  version = {2026-10-29.1},
  organization = {NUWA Frontier AI Safety Lab},
  howpublished = {Retained note edition: https://whitzard.tech/nuwa/whitzard-index/data/record-editions/2026-10-29.1--extreme-risk-evaluations.json},
  note = {NUWA research note. Source-reading scope and independent review status are stated on the record. Snapshot 2026-10-29.1. Retained note edition: https://whitzard.tech/nuwa/whitzard-index/data/record-editions/2026-10-29.1--extreme-risk-evaluations.json.}
}

Evidence snapshot 2026-10-29.1 · Index updates · RSS

References

[1] Toby Shevlane, Sebastian Farquhar, et al.. Model evaluation for extreme risks. 2023-05-24. v2 [arXiv]; Author affiliations on the original paper title page; 2 Extreme risks from general-purpose models; 3 Model evaluation as critical governance infrastructure