Risk Evaluation · Benchmark
The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearning
NUWA research note · independent review pending
NUWA Research Note
Research questionDesign and assumptionsSource findingsNUWA interpretationLimits and contrary evidenceResearch and decision relevanceInstitutions and rolesOriginal resourcesModel-assisted reading · Selected sections of the original full text or official article; a complete section-by-section audit remains pending.
At a glance
NUWA interpretation: proxy knowledge accuracy measures a narrower construct than operational harm or user uplift.
Research question
Can a public proxy benchmark track hazardous knowledge while supporting mitigation research?
Design and assumptions
WMDP contains 3,668 expert-written multiple-choice items across three risk domains, filtered to exclude sensitive and export-controlled information.
Source findings
The paper evaluates knowledge-reduction methods alongside general-capability retention.
Reported mitigation gains also affect some adjacent benign subjects, revealing a precision trade-off (§7).
NUWA interpretation
NUWA interpretation: proxy knowledge accuracy measures a narrower construct than operational harm or user uplift.
Limits and contrary evidence
A low multiple-choice score does not by itself establish durable removal of knowledge or reduced real-world misuse.
Research and decision relevance
Model recommendation: evaluate safeguard durability and benign utility separately; retain only nonoperational benchmark statistics in this index.
Precise source locations
- The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearning · AbstractSelected original resource https://arxiv.org/html/2403.03218v7; sha256:472585ab0b6df3fe4f74bee849a6a68087e4641617a5c8914c0b057a27ca3d78; archived source capture; exact capture time not independently established
- The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearning · §3 The WMDP BenchmarkSelected original resource https://arxiv.org/html/2403.03218v7; sha256:472585ab0b6df3fe4f74bee849a6a68087e4641617a5c8914c0b057a27ca3d78; archived source capture; exact capture time not independently established
- The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearning · §5 Experimental ResultsSelected original resource https://arxiv.org/html/2403.03218v7; sha256:472585ab0b6df3fe4f74bee849a6a68087e4641617a5c8914c0b057a27ca3d78; archived source capture; exact capture time not independently established
- The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearning · §7 ConclusionSelected original resource https://arxiv.org/html/2403.03218v7; sha256:472585ab0b6df3fe4f74bee849a6a68087e4641617a5c8914c0b057a27ca3d78; archived source capture; exact capture time not independently established
Source checking, extraction, editorial approval and replication are recorded separately. Methodology →
Institutions and roles
Authors (57)
Nathaniel Li · Alexander Pan · Anjali Gopal · Summer Yue · Daniel Berrios · Alice Gatti · Justin D. Li · Ann-Kathrin Dombrowski · Shashwat Goel · Long Phan · Gabriel Mukobi · Nathan Helm-Burger · Rassin Lababidi · Lennart Justen · Andrew B. Liu · Michael Chen · Isabelle Barrass · Oliver Zhang · Xiaoyuan Zhu · Rishub Tamirisa · Bhrugu Bharathi · Adam Khoja · Zhenqi Zhao · Ariel Herbert-Voss · Cort B. Breuer · Samuel Marks · Oam Patel · Andy Zou · Mantas Mazeika · Zifan Wang · Palash Oswal · Weiran Lin · Adam A. Hunt · Justin Tienken-Harder · Kevin Y. Shih · Kemper Talley · John Guan · Russell Kaplan · Ian Steneker · David Campbell · Brad Jokubaitis · Alex Levinson · Jean Wang · William Qian · Kallol Krishna Karmakar · Steven Basart · Stephen Fitz · Mindy Levine · Ponnurangam Kumaraguru · Uday Tupakula · Vijay Varadharajan · Ruoyu Wang · Yan Shoshitaishvili · Jimmy Ba · Kevin M. Esvelt · Alexandr Wang · Dan Hendrycks
Evaluation protocols
Cite and reuse
Cite the original publication for its findings. Cite this NUWA record separately for its explanation and retained snapshot.
Original publication · BibTeX
@misc{wmdp,
title = {The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearning},
url = {https://arxiv.org/abs/2403.03218v7},
author = {Nathaniel Li and Alexander Pan and Anjali Gopal and Summer Yue and Daniel Berrios and Alice Gatti and Justin D. Li and Ann-Kathrin Dombrowski and Shashwat Goel and Long Phan and Gabriel Mukobi and Nathan Helm-Burger and Rassin Lababidi and Lennart Justen and Andrew B. Liu and Michael Chen and Isabelle Barrass and Oliver Zhang and Xiaoyuan Zhu and Rishub Tamirisa and Bhrugu Bharathi and Adam Khoja and Zhenqi Zhao and Ariel Herbert-Voss and Cort B. Breuer and Samuel Marks and Oam Patel and Andy Zou and Mantas Mazeika and Zifan Wang and Palash Oswal and Weiran Lin and Adam A. Hunt and Justin Tienken-Harder and Kevin Y. Shih and Kemper Talley and John Guan and Russell Kaplan and Ian Steneker and David Campbell and Brad Jokubaitis and Alex Levinson and Jean Wang and William Qian and Kallol Krishna Karmakar and Steven Basart and Stephen Fitz and Mindy Levine and Ponnurangam Kumaraguru and Uday Tupakula and Vijay Varadharajan and Ruoyu Wang and Yan Shoshitaishvili and Jimmy Ba and Kevin M. Esvelt and Alexandr Wang and Dan Hendrycks},
year = {2024},
date = {2024-03-05},
version = {arXiv:2403.03218v7},
note = {arXiv:2403.03218v7}
}
NUWA record · BibTeX
@misc{wmdp-nuwa,
title = {The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearning — NUWA research record},
url = {https://whitzard.tech/en/nuwa/whitzard-index/work/wmdp/},
version = {2026-10-29.1},
organization = {NUWA Frontier AI Safety Lab},
howpublished = {Retained note edition: https://whitzard.tech/nuwa/whitzard-index/data/record-editions/2026-10-29.1--wmdp.json},
note = {NUWA research note. Source-reading scope and independent review status are stated on the record. Snapshot 2026-10-29.1. Retained note edition: https://whitzard.tech/nuwa/whitzard-index/data/record-editions/2026-10-29.1--wmdp.json.}
}
Evidence snapshot 2026-10-29.1 · Index updates · RSS
References
- [1] Nathaniel Li ∗, Alexander Pan ∗, et al.. The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearning. 2024-05-15. v7 [arXiv]; Author affiliations on the original paper title page; Abstract; §3 The WMDP Benchmark