NUWA Risk questions
CBRN
What assistance could AI provide across hazardous scientific knowledge and consequential task steps?
CBRN
Scientific capabilities, misuse pathways and the reach of safeguards.
Scientific skills and misuse evidence
New research and interpretation
Selected original sections · model-assisted reading; independent review pending. These records add context without changing trend scores.
- Frontier AI Trends Report →
Cross-domain trends require methodology-specific interpretation and negative-evidence limits.
- Long-Form Tasks: A Methodology for Evaluating Scientific Assistants →
Long-form evaluation is a bridge toward human-utility measurement, not a replacement for uplift trials.
- ReproBAIT: Can coding agents build their own biological AI tools? →
Reconstruction fidelity and memorisation need separate treatment in tool-access governance.
- The Effectiveness and Durability of Pretraining Data Filtering in Protein Language Models →
Initial suppression and durable safeguards are different claims.
Scope
Track knowledge probes separately from practical task uplift; publish only bounded, non-operational summaries.
Threat model
A user with defined expertise and resources seeks help for a hazardous task.
Evidence gaps
Multiple-choice knowledge does not prove end-to-end harmful capability.
Research on this topic
The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearning
Center for AI Safety
University of California, Berkeley
Massachusetts Institute of Technology
SecureBio
Scale AI
New York University
IIIT Hyderabad
Stanford University
Harvard University
University of Southern California
University of Illinois Urbana-ChampaignLapis Labs
University of California, Los Angeles
California Institute of TechnologySybil
Northeastern University
Carnegie Mellon University
RTX BBN Technologies
University of Newcastle
Keio University
Ariel University
Arizona State University
xAI
QuestionCan a public proxy benchmark track hazardous knowledge while supporting mitigation research?
Source summaryThe paper evaluates knowledge-reduction methods alongside general-capability retention.
NUWA research note · independent review pending · Contains: question / method / limits
Virology Capabilities Test (VCT): A Multimodal Virology Q&A Benchmark
QuestionHow well do models answer expert-authored virology troubleshooting questions?
Source summaryo3 scored 43.8%, versus a 22.1% mean for expert virologists with internet access.
NUWA research note · independent review pending · Contains: question / method / interpretation / limits
Introducing VCT-v2 — the updated Virology Capabilities Test
QuestionCan revisions improve VCT’s answer-key reliability and resistance to shortcuts?
Source summaryVCT-v2 has 279 questions; the top reported model scored 49.2%.
NUWA research note · independent review pending · Contains: question / method / interpretation / limits
BioTIER: A Refusal Benchmark for Targeted Biological Risk Mitigation
QuestionCan biological safeguards distinguish narrowly high-risk requests from benign scientific access?
Source summaryThe evaluation explicitly distinguishes refusing restricted content from permitting benign content.
NUWA research note · independent review pending · Contains: question / method / interpretation / limits
LABBench2: An Improved Benchmark for AI Systems Performing Biology Research
QuestionCan scientific capability tests better approximate useful research work than multiple-choice knowledge questions?
Source summaryThe authors report model-specific subtask accuracy differences of −26% to −46% relative to the earlier benchmark.
NUWA research note · independent review pending
FORTRESS: Frontier Risk Evaluation for National Security and Public Safety
QuestionHow do safeguards balance national/public-safety risk restriction against benign access?
Source summaryClaude 3.5 Sonnet’s reported ARS/ORS were 14.09/21.80, versus Gemini 2.5 Pro’s 66.29/1.40.
NUWA research note · independent review pending
Evidence matrix
Rows are research questions; columns show what published evaluations observe. Counts refer to evaluation dossiers, not risk levels.
| Risk questions | Propensity | Capability | Behavior |
|---|---|---|---|
| Hazardous knowledgeWhat hazardous knowledge can a model recall or apply on bounded questions? | —No evidence in the current catalogue | 3evaluation dossiers | 1evaluation dossiers |
| Task upliftDoes AI assistance improve a specified actor’s performance on a hazardous task? | —No evidence in the current catalogue | 2evaluation dossiers | —No evidence in the current catalogue |
The paths show research relationships, not proven causal chains or a risk score.
CBRN / Propensity
Hazardous knowledge
What hazardous knowledge can a model recall or apply on bounded questions?
- Included
- Knowledge questions with explicit subject and scoring.
- Does not establish
- Knowledge scores do not prove experimental execution.
No evidence in the current catalogue
CBRN / Capability
Hazardous knowledge
What hazardous knowledge can a model recall or apply on bounded questions?
- Included
- Knowledge questions with explicit subject and scoring.
- Does not establish
- Knowledge scores do not prove experimental execution.
3 evaluation dossiers · 26 verified result records
WMDP ↗
Center for AI SafetyUniversity of California, BerkeleyMassachusetts Institute of TechnologySecureBioScale AINew York UniversityIIIT HyderabadStanford UniversityHarvard UniversityUniversity of Southern CaliforniaUniversity of Illinois Urbana-ChampaignLapis LabsUniversity of California, Los AngelesCalifornia Institute of TechnologySybilNortheastern UniversityCarnegie Mellon UniversityRTX BBN TechnologiesUniversity of NewcastleKeio UniversityAriel UniversityArizona State UniversityxAI
The source directly measures the stated construct under its protocol.
Assessment subject: Model answering hazardous knowledge questions
Evidence relationship: arXiv:2403.03218v7, multiple-choice task design
Virology Capabilities Test (VCT) ↗
The source tests domain knowledge and troubleshooting as a proxy, not end-to-end task execution.
Assessment subject: Models in specialist virology question tests
Evidence relationship: arXiv:2504.16137v2, abstract and test design
Virology Capabilities Test v2 ↗
The source tests domain knowledge and troubleshooting as a proxy, not end-to-end task execution.
Assessment subject: Models in specialist virology question tests
Evidence relationship: SecureBio, Presenting our updated benchmark
CBRN / Behavior
Hazardous knowledge
What hazardous knowledge can a model recall or apply on bounded questions?
- Included
- Knowledge questions with explicit subject and scoring.
- Does not establish
- Knowledge scores do not prove experimental execution.
1 evaluation dossiers · 0 verified result records
BioTIER ↗
Response handling supplies safeguard context for this risk topic, not a hazardous-capability score.
Assessment subject: Models receiving biological-risk prompts
Evidence relationship: arXiv:2607.14479v1, abstract and risk sets
CBRN / Propensity
Task uplift
Does AI assistance improve a specified actor’s performance on a hazardous task?
- Included
- High-level, non-operational uplift study metadata.
- Does not establish
- Answer quality alone does not establish uplift.
No evidence in the current catalogue
CBRN / Capability
Task uplift
Does AI assistance improve a specified actor’s performance on a hazardous task?
- Included
- High-level, non-operational uplift study metadata.
- Does not establish
- Answer quality alone does not establish uplift.
2 evaluation dossiers · 0 verified result records
LAB-Bench ↗
The protocol explicitly measures this question or an enabling sub-capability; conclusions retain its stated conditions.
Assessment subject: The model or agent in the stated task configuration.
Evidence relationship: Original protocol method and limitations
BioMysteryBench ↗
The protocol explicitly measures this question or an enabling sub-capability; conclusions retain its stated conditions.
Assessment subject: The model or agent in the stated task configuration.
Evidence relationship: Original protocol method and limitations
CBRN / Behavior
Task uplift
Does AI assistance improve a specified actor’s performance on a hazardous task?
- Included
- High-level, non-operational uplift study metadata.
- Does not establish
- Answer quality alone does not establish uplift.
No evidence in the current catalogue
References
- [1] Evaluating DeepSeek v4 Pro for Frontier Risks. 2026-05-29. v1.1 · 29 May 2026 ; §2.1; §2.2
- [2] GLM-5.2 Risk Evaluation Report. 2026-08-02. v1.1 · revised 26 August 2026 ; §3.1; Appendix A.3; §3.2; Appendix A.4
- [3] Grok 4.7 Model Card. 2026-09-21 ; PDF pp.20–23 (printed 19–22)
- [4] Nathaniel Li ∗, Alexander Pan ∗, et al.. The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearning. 2024-05-15. v7 ; arXiv:2403.03218v7, multiple-choice task design
- [5] Evaluating Frontier Models for Dangerous Capabilities. 2024-03-21 publication
- [6] OpenAI Deployment Safety Hub. Index checked 2026-10-25
- [7] Muse Spark Safety & Preparedness Report. arXiv
- [8] Jasper Götting a, Pedro Medeiros a, et al.. Virology Capabilities Test (VCT): A Multimodal Virology Q&A Benchmark. 2025-04-29. v2 ; arXiv:2504.16137v2, abstract and test design
- [9] Introducing VCT-v2 — the updated Virology Capabilities Test. 2026-09-11 ; SecureBio, Presenting our updated benchmark
- [10] Eleanor M. Marshall, Pedro Medeiros, et al.. BioTIER: A Refusal Benchmark for Targeted Biological Risk Mitigation. 2026-07-16. v1 ; arXiv:2607.14479v1, abstract and risk sets
- [11] LAB-Bench. 2024-07-14 ; Original protocol method and limitations
- [12] BioMysteryBench ; Original protocol method and limitations