{"schemaVersion":"nuwa-current-record-1","version":"2026-10-29.1","publication":"published","work":{"id":"lab-bench","title":"LAB-Bench Measuring Capabilities of Language Models for Biology Research","aliases":["LAB-Bench"],"authors":[],"primaryFunction":"evaluation","secondaryFunctions":[],"kind":"benchmark","sourceId":"lab-bench-primary","attributionSourceId":"lab-bench-primary","originalUrl":"https://arxiv.org/abs/2407.10362","publicationDate":"2024-07-14","updatedOn":"2026-10-25","areas":["cbrn"],"scopeNote":{"en":"Mapped to the documented research question. A risk-topic association is not a severity score.","zh":"依据明确研究问题关联；风险主题不代表严重性评分。"},"protocolIds":["lab-bench"],"contributions":[],"summary":{"en":"How do models perform on research-oriented scientific tasks?","zh":"模型在面向研究的科学任务中表现如何？"},"note":{"question":{"en":"How do models perform on research-oriented scientific tasks?","zh":"模型在面向研究的科学任务中表现如何？"},"method":{"en":"Evaluates a range of biology research skills with structured questions; text-only or missing-input subsets remain separate protocols.","zh":"用结构化问题评测生物研究技能；纯文本或缺失输入子集采用独立协议。"},"finding":null,"supports":null,"limits":{"en":"These are general scientific capabilities, not end-to-end misuse tasks or direct harm probabilities.","zh":"衡量一般科研能力，不是端到端滥用任务或直接伤害概率。"},"connections":null},"noteAvailability":{"en":"This record describes the protocol. Report-specific outcomes are linked through assessments; no unsourced general finding is added.","zh":"本档案说明协议。报告特定结果通过评估记录关联，不补写无出处的总体发现。"},"resources":[],"tags":[]},"reading":{"id":"lab-bench","title":"LAB-Bench Measuring Capabilities of Language Models for Biology Research","en":{"question":"Can biology benchmarks move beyond textbook knowledge toward practical research-support skills?","design":"LAB-Bench contains over 2,400 multiple-choice questions spanning literature, figures, tables, database access and other research-support tasks, with partial expert baselines.","findings":["The construction process found expert revision of model-proposed questions useful.","Expert human coverage was incomplete because recruiting qualified baseliners for the entire set was infeasible (§3.5)."],"interpretation":"NUWA interpretation: task-format realism and human-baseline coverage determine what comparisons mean.","limits":"The authors do not claim comprehensive biology coverage; unassisted multiple-choice results do not establish real laboratory execution or uplift.","decisionRelevance":"Model recommendation: retain subtask, tools and actual human coverage, without extracting operational biological content.","takeaway":"NUWA interpretation: task-format realism and human-baseline coverage determine what comparisons mean.","connections":[],"scoring":"Multiple-choice research-support accuracy, with subtask-specific and incomplete human baselines.","conditions":"More than 2,400 tasks; no operational wet-lab execution measurement.","statistics":"Expert human coverage was incomplete because recruiting qualified baseliners for the entire set was infeasible (§3.5).","usage":"Model recommendation: retain subtask, tools and actual human coverage, without extracting operational biological content."},"zh":{"question":"生物基准能否从教科书知识转向实际研究支持技能？","design":"LAB-Bench含逾2,400道选择题，覆盖文献、图表、数据库等研究支持任务，配部分专家基线。","findings":["构造中发现专家修改模型提议题目有帮助。","因招募困难，人类专家未覆盖全题集（§3.5）。"],"interpretation":"NUWA 解读：格式真实性及人类覆盖决定比较含义。","limits":"作者不声称全面覆盖生物，无工具选择题不能确立真实实验执行或增益。","decisionRelevance":"模型建议：保留子任务、工具及真实人类覆盖，不提取操作性生物内容。","takeaway":"NUWA 解读：格式真实性及人类覆盖决定比较含义。","connections":[],"scoring":"研究支持选择题正确率，分子任务且人类基线覆盖不全。","conditions":"逾2,400题，不测操作性湿实验执行。","statistics":"因招募困难，人类专家未覆盖全题集（§3.5）。","usage":"模型建议：保留子任务、工具及真实人类覆盖，不提取操作性生物内容。"},"assertionPolicy":{"question":"source-research-question","design":"source-finding","findings":"source-finding","interpretation":"nuwa-interpretation","limits":"source-limit-or-explicit-editorial-scope-limit","connections":"source-supported-relationship-only","decisionRelevance":"model-recommendation-not-source-conclusion"},"sourceRefs":[{"sourceId":"lab-bench-primary","url":"https://arxiv.org/html/2407.10362","locator":"Abstract","version":"Selected original resource https://arxiv.org/html/2407.10362; sha256:fb4550ba8cc17a4ffb2f30a3e9f1fe78a538a5c1d3508550942723e506ffe2c8; archived source capture; exact capture time not independently established","artifactSha256":"fb4550ba8cc17a4ffb2f30a3e9f1fe78a538a5c1d3508550942723e506ffe2c8"},{"sourceId":"lab-bench-primary","url":"https://arxiv.org/html/2407.10362","locator":"§2.1 Dataset description and construction","version":"Selected original resource https://arxiv.org/html/2407.10362; sha256:fb4550ba8cc17a4ffb2f30a3e9f1fe78a538a5c1d3508550942723e506ffe2c8; archived source capture; exact capture time not independently established","artifactSha256":"fb4550ba8cc17a4ffb2f30a3e9f1fe78a538a5c1d3508550942723e506ffe2c8"},{"sourceId":"lab-bench-primary","url":"https://arxiv.org/html/2407.10362","locator":"§3 Discussion and Limitations","version":"Selected original resource https://arxiv.org/html/2407.10362; sha256:fb4550ba8cc17a4ffb2f30a3e9f1fe78a538a5c1d3508550942723e506ffe2c8; archived source capture; exact capture time not independently established","artifactSha256":"fb4550ba8cc17a4ffb2f30a3e9f1fe78a538a5c1d3508550942723e506ffe2c8"},{"sourceId":"lab-bench-primary","url":"https://arxiv.org/html/2407.10362","locator":"§3.5 Limitations in human evaluation","version":"Selected original resource https://arxiv.org/html/2407.10362; sha256:fb4550ba8cc17a4ffb2f30a3e9f1fe78a538a5c1d3508550942723e506ffe2c8; archived source capture; exact capture time not independently established","artifactSha256":"fb4550ba8cc17a4ffb2f30a3e9f1fe78a538a5c1d3508550942723e506ffe2c8"}],"publication":{"state":"published","publishedOn":"2026-10-29","authorizationReference":"user-authorized-global-index-publication-2026-10-29","reviewedCandidateId":"local-6efe31bc4a0c0f37"},"review":{"state":"published-note","sourceScope":"full-text","availableScope":null,"readingCoverage":"selected-source-sections; not a complete section-by-section paper audit","completePaperAudit":false,"independentEvidenceReview":"pending-human-review","bilingualEditorialReview":"pending-human-review","authorship":"model-assisted-research-note","preparedFingerprint":"d8a14d1cb0e8d5d0c0b4ca31d234cd88cfbee1a8b3f06f2b15f343a28b01064d"}},"sources":[{"id":"lab-bench-primary","title":"LAB-Bench","url":"https://arxiv.org/abs/2407.10362","version":"2026-10-25 source capture","publicationDate":"2024-07-14","datePrecision":"day","type":"original-report","verifiedOn":"2026-10-25","authors":[],"documentId":"arxiv:2407.10362"}]}