Risk Evaluation · Technical report

Toward Comprehensive Benchmarking of the Biological Knowledge of Frontier Large Language Models

NUWA research note · independent review pending

First published: 2025-11-25Source checked: 2026-10-25

Original source ↗Cite this record ↓CBRN →

NUWA Research Note

Research question Design and assumptions Source findings NUWA interpretation Limits and contrary evidence Research and decision relevance Institutions and roles Original resources

Model-assisted reading · Selected sections of the original full text or official article; a complete section-by-section audit remains pending.

At a glance

NUWA interpretation: knowledge scores, refusal robustness and real-world uplift require separate evidence.

Research question

What can existing biological/chemical knowledge and refusal benchmarks establish about frontier models?

Design and assumptions

The official report summary compares models available by May 2025 on eight knowledge/refusal benchmarks and discusses human-baseline design.

Source findings

The authors identify widespread benchmark saturation and frontier reasoning-model performance above several expert-question baselines.

Reducing refusals through the studied tuning also reduced some knowledge performance; risk effects remained unclear.

NUWA interpretation

NUWA interpretation: knowledge scores, refusal robustness and real-world uplift require separate evidence.

Limits and contrary evidence

This reading covers the official overview, key takeaways and recommendations, not a complete report-methods audit.

Research and decision relevance

Model recommendation: prioritize qualified domain-matched human baselines, harder held-out items and implementation disclosure.

Precise source locations

Toward Comprehensive Benchmarking of the Biological Knowledge of Frontier Large Language Models · Report overviewSelected original resource https://www.rand.org/pubs/research_reports/RRA3797-1.html; sha256:3a110b6e4b9c0f8c321b3c01015fa4ddc3052614e0585722c3c73cbc964f459b; archived source capture; exact capture time not independently established
Toward Comprehensive Benchmarking of the Biological Knowledge of Frontier Large Language Models · Key TakeawaysSelected original resource https://www.rand.org/pubs/research_reports/RRA3797-1.html; sha256:3a110b6e4b9c0f8c321b3c01015fa4ddc3052614e0585722c3c73cbc964f459b; archived source capture; exact capture time not independently established
Toward Comprehensive Benchmarking of the Biological Knowledge of Frontier Large Language Models · RecommendationsSelected original resource https://www.rand.org/pubs/research_reports/RRA3797-1.html; sha256:3a110b6e4b9c0f8c321b3c01015fa4ddc3052614e0585722c3c73cbc964f459b; archived source capture; exact capture time not independently established

Source checking, extraction, editorial approval and replication are recorded separately. Methodology →

Institutions and roles

Published by RAND

Authors (23)

Sunishchal Dev · Charles Teague · Grant Ellison · Kyle Brady · Jeffrey Lee · Sarah L. Gebauer · Henry Alexander Bradley · Dawid Maciorowski · Bria Persaud · Jordan Despanie · Barbara Del Castello · Alyssa Worland · Michael Miller · Adrian Salas · Dave Nguyen · James Liu · Jason Johnson · Andrew Sloan · Will Stonehouse · Travis Merrill · Thomas Goode · McKelvey, Greg, Jr. · Ella Guest

Cite and reuse

Cite the original publication for its findings. Cite this NUWA record separately for its explanation and retained snapshot.

BibTeX ↓RIS ↓Record + source metadata · JSON ↓Reading coverage · JSON Suggest a correction

Original publication · BibTeX

@misc{rand-biological-benchmarking,
  title = {Toward Comprehensive Benchmarking of the Biological Knowledge of Frontier Large Language Models},
  url = {https://www.rand.org/pubs/research_reports/RRA3797-1.html},
  author = {Sunishchal Dev and Charles Teague and Grant Ellison and Kyle Brady and Jeffrey Lee and Sarah L. Gebauer and Henry Alexander Bradley and Dawid Maciorowski and Bria Persaud and Jordan Despanie and Barbara Del Castello and Alyssa Worland and Michael Miller and Adrian Salas and Dave Nguyen and James Liu and Jason Johnson and Andrew Sloan and Will Stonehouse and Travis Merrill and Thomas Goode and McKelvey, Greg, Jr. and Ella Guest},
  year = {2025},
  date = {2025-11-25},
  version = {Source capture 2026-10-25},
  note = {Source capture 2026-10-25}
}

NUWA record · BibTeX

@misc{rand-biological-benchmarking-nuwa,
  title = {Toward Comprehensive Benchmarking of the Biological Knowledge of Frontier Large Language Models — NUWA research record},
  url = {https://whitzard.tech/en/nuwa/whitzard-index/work/rand-biological-benchmarking/},
  version = {2026-10-29.1},
  organization = {NUWA Frontier AI Safety Lab},
  howpublished = {Retained note edition: https://whitzard.tech/nuwa/whitzard-index/data/record-editions/2026-10-29.1--rand-biological-benchmarking.json},
  note = {NUWA research note. Source-reading scope and independent review status are stated on the record. Snapshot 2026-10-29.1. Retained note edition: https://whitzard.tech/nuwa/whitzard-index/data/record-editions/2026-10-29.1--rand-biological-benchmarking.json.}
}

Evidence snapshot 2026-10-29.1 · Index updates · RSS

References

[1] Dev, Sunishchal, Teague, Charles, et al.. Toward Comprehensive Benchmarking of the Biological Knowledge of Frontier Large Language Models. 2025-11-25 ; Official publication masthead and article byline; Report overview; Key Takeaways