Risk Evaluation · Study

Evaluating Frontier Models for Dangerous Capabilities

NUWA research note · independent review pending

First published: 2024-03-20Source checked: 2026-10-25

Original source ↗Cite this record ↓Autonomy →Deception →Cybersecurity →

NUWA Research Note

Research question Design and assumptions Source findings NUWA interpretation Limits and contrary evidence Research and decision relevance Institutions and roles Original resources

Model-assisted reading · Selected sections of the original full text or official article; a complete section-by-section audit remains pending.

At a glance

NUWA interpretation: bounded negative findings depend on both task coverage and elicitation.

Research question

What warning signs can a structured dangerous-capability evaluation program reveal?

Design and assumptions

Gemini 1.0 pilots span persuasion/deception, cyber, self-proliferation and self-reasoning, with multiple tasks and elicitation approaches.

Source findings

The authors found no strong dangerous capabilities in tested models, while identifying early warning signs.

Vulnerability-patch classification accuracy differed substantially between datasets (§5.2).

NUWA interpretation

NUWA interpretation: bounded negative findings depend on both task coverage and elicitation.

Limits and contrary evidence

The program is an early pilot on specific model generations, not an enduring guarantee against future dangerous capabilities.

Research and decision relevance

Model recommendation: use the domain/task decomposition as an audit design reference and date every capability claim.

Precise source locations

Evaluating Frontier Models for Dangerous Capabilities · AbstractSelected original resource https://arxiv.org/html/2403.13793; sha256:12872dcaaa7ad2297665fd85c8404512e050c28e5616ce15c854f536ea0fd945; archived source capture; exact capture time not independently established
Evaluating Frontier Models for Dangerous Capabilities · §2 Overview of evaluations and agentsSelected original resource https://arxiv.org/html/2403.13793; sha256:12872dcaaa7ad2297665fd85c8404512e050c28e5616ce15c854f536ea0fd945; archived source capture; exact capture time not independently established
Evaluating Frontier Models for Dangerous Capabilities · §5.2 ResultsSelected original resource https://arxiv.org/html/2403.13793; sha256:12872dcaaa7ad2297665fd85c8404512e050c28e5616ce15c854f536ea0fd945; archived source capture; exact capture time not independently established
Evaluating Frontier Models for Dangerous Capabilities · §6.4 DiscussionSelected original resource https://arxiv.org/html/2403.13793; sha256:12872dcaaa7ad2297665fd85c8404512e050c28e5616ce15c854f536ea0fd945; archived source capture; exact capture time not independently established

Source checking, extraction, editorial approval and replication are recorded separately. Methodology →

Institutions and roles

Research by Google DeepMind

Authors (27)

Mary Phuong · Matthew Aitchison · Elliot Catt · Sarah Cogan · Alexandre Kaskasoli · Victoria Krakovna · David Lindner · Matthew Rahtz · Yannis Assael · Sarah Hodkinson · Heidi Howard · Tom Lieberum · Ramana Kumar · Maria Abi Raad · Albert Webson · Lewis Ho · Sharon Lin · Sebastian Farquhar · Marcus Hutter · Gregoire Deletang · Anian Ruoss · Seliem El-Sayed · Sasha Brown · Anca Dragan · Rohin Shah · Allan Dafoe · Toby Shevlane

Cite and reuse

Cite the original publication for its findings. Cite this NUWA record separately for its explanation and retained snapshot.

BibTeX ↓RIS ↓Record + source metadata · JSON ↓Reading coverage · JSON Suggest a correction

Original publication · BibTeX

@misc{deepmind-dangerous-capabilities,
  title = {Evaluating Frontier Models for Dangerous Capabilities},
  url = {https://arxiv.org/abs/2403.13793},
  author = {Mary Phuong and Matthew Aitchison and Elliot Catt and Sarah Cogan and Alexandre Kaskasoli and Victoria Krakovna and David Lindner and Matthew Rahtz and Yannis Assael and Sarah Hodkinson and Heidi Howard and Tom Lieberum and Ramana Kumar and Maria Abi Raad and Albert Webson and Lewis Ho and Sharon Lin and Sebastian Farquhar and Marcus Hutter and Gregoire Deletang and Anian Ruoss and Seliem El-Sayed and Sasha Brown and Anca Dragan and Rohin Shah and Allan Dafoe and Toby Shevlane},
  year = {2024},
  date = {2024-03-20},
  version = {arXiv:2403.13793v2},
  note = {arXiv:2403.13793v2}
}

NUWA record · BibTeX

@misc{deepmind-dangerous-capabilities-nuwa,
  title = {Evaluating Frontier Models for Dangerous Capabilities — NUWA research record},
  url = {https://whitzard.tech/en/nuwa/whitzard-index/work/deepmind-dangerous-capabilities/},
  version = {2026-10-29.1},
  organization = {NUWA Frontier AI Safety Lab},
  howpublished = {Retained note edition: https://whitzard.tech/nuwa/whitzard-index/data/record-editions/2026-10-29.1--deepmind-dangerous-capabilities.json},
  note = {NUWA research note. Source-reading scope and independent review status are stated on the record. Snapshot 2026-10-29.1. Retained note edition: https://whitzard.tech/nuwa/whitzard-index/data/record-editions/2026-10-29.1--deepmind-dangerous-capabilities.json.}
}

Evidence snapshot 2026-10-29.1 · Index updates · RSS

References

[1] Mary Phuong, Matthew Aitchison, et al.. Evaluating Frontier Models for Dangerous Capabilities. 2024-03-20. v2 [arXiv]; Abstract; §2 Overview of evaluations and agents; §5.2 Results
[2] Evaluating Frontier Models for Dangerous Capabilities. 2024-03-21 ; Official research publication and author list