NUWA Risk Evaluation · Benchmark
LAB-Bench Measuring Capabilities of Language Models for Biology Research
NUWA research note · independent review pending
NUWA Research Note
Research questionDesign and assumptionsSource findingsNUWA interpretationLimits and contrary evidenceResearch and decision relevanceInstitutions and rolesOriginal resourcesModel-assisted reading · Selected sections of the original full text or official article; a complete section-by-section audit remains pending.
At a glance
NUWA interpretation: task-format realism and human-baseline coverage determine what comparisons mean.
Research question
Can biology benchmarks move beyond textbook knowledge toward practical research-support skills?
Design and assumptions
LAB-Bench contains over 2,400 multiple-choice questions spanning literature, figures, tables, database access and other research-support tasks, with partial expert baselines.
Source findings
The construction process found expert revision of model-proposed questions useful.
Expert human coverage was incomplete because recruiting qualified baseliners for the entire set was infeasible (§3.5).
NUWA interpretation
NUWA interpretation: task-format realism and human-baseline coverage determine what comparisons mean.
Limits and contrary evidence
The authors do not claim comprehensive biology coverage; unassisted multiple-choice results do not establish real laboratory execution or uplift.
Research and decision relevance
Model recommendation: retain subtask, tools and actual human coverage, without extracting operational biological content.
Precise source locations
- LAB-Bench · AbstractSelected original resource https://arxiv.org/html/2407.10362; sha256:fb4550ba8cc17a4ffb2f30a3e9f1fe78a538a5c1d3508550942723e506ffe2c8; archived source capture; exact capture time not independently established
- LAB-Bench · §2.1 Dataset description and constructionSelected original resource https://arxiv.org/html/2407.10362; sha256:fb4550ba8cc17a4ffb2f30a3e9f1fe78a538a5c1d3508550942723e506ffe2c8; archived source capture; exact capture time not independently established
- LAB-Bench · §3 Discussion and LimitationsSelected original resource https://arxiv.org/html/2407.10362; sha256:fb4550ba8cc17a4ffb2f30a3e9f1fe78a538a5c1d3508550942723e506ffe2c8; archived source capture; exact capture time not independently established
- LAB-Bench · §3.5 Limitations in human evaluationSelected original resource https://arxiv.org/html/2407.10362; sha256:fb4550ba8cc17a4ffb2f30a3e9f1fe78a538a5c1d3508550942723e506ffe2c8; archived source capture; exact capture time not independently established
Source checking, extraction, editorial approval and replication are recorded separately. Methodology →
Institutions and roles
Evaluation protocols
Cite and reuse
Cite the original publication for its findings. Cite this NUWA record separately for its explanation and retained snapshot.
Original publication · BibTeX
@misc{lab-bench,
title = {LAB-Bench Measuring Capabilities of Language Models for Biology Research},
url = {https://arxiv.org/abs/2407.10362},
year = {2024},
date = {2024-07-14},
version = {2026-10-25 source capture},
note = {2026-10-25 source capture}
}
NUWA record · BibTeX
@misc{lab-bench-nuwa,
title = {LAB-Bench Measuring Capabilities of Language Models for Biology Research — NUWA research record},
url = {https://whitzard.tech/en/nuwa/whitzard-index/work/lab-bench/},
version = {2026-10-29.1},
organization = {NUWA Frontier AI Safety Lab},
howpublished = {Retained note edition: https://whitzard.tech/nuwa/whitzard-index/data/record-editions/2026-10-29.1--lab-bench.json},
note = {NUWA research note. Source-reading scope and independent review status are stated on the record. Snapshot 2026-10-29.1. Retained note edition: https://whitzard.tech/nuwa/whitzard-index/data/record-editions/2026-10-29.1--lab-bench.json.}
}
Evidence snapshot 2026-10-29.1 · Index updates · RSS
References
- [1] LAB-Bench. 2024-07-14 ; Abstract; §2.1 Dataset description and construction; §3 Discussion and Limitations