Evaluation dossier

LAB-Bench

How do models perform on research-oriented scientific tasks?

SaferAI / 2026-10-25 source capture / Capability

Sources and provenance ↓ Read the research narrative →

Task and measurement

Benchmark protocol profile

Task / subject: The model or agent in the stated task configuration.
Measurement target: Evaluates a range of biology research skills with structured questions; text-only or missing-input subsets remain separate protocols.
Scoring / outcome: FigQA attempt, figure withheld · Evaluates a range of biology research skills with structured questions; text-only or missing-input subsets remain separate protocols.
Interpretation boundary: These are general scientific capabilities, not end-to-end misuse tasks or direct harm probabilities.

SaferAI evaluates the public API without developer cooperation, combining its own benchmark runs with explicitly identified external comparison results. The report separates task capability, adversarially elicited behavior and willingness to attempt persuasion; configurations and provenance vary by benchmark.

A reported evaluation

The source evaluates Kimi K3 · LAB-Bench with figures/tables withheld.

Reported measurement: FigQA attempt, figure withheld · 79.6%.

Evaluates a range of biology research skills with structured questions; text-only or missing-input subsets remain separate protocols. Figure/table subsets with explicit insufficient-information answer option.

Source-reported measurements in §1.1.2; Figure 2. Transcription is not independent experimental replication.

Measurement dossier · NUWA reading

Source-method reading and retained catalogue information are distinguished. Publication does not establish independent evidence or bilingual editorial review.

Measurement construct: Can biology benchmarks move beyond textbook knowledge toward practical research-support skills?
Task population: LAB-Bench contains over 2,400 multiple-choice questions spanning literature, figures, tables, database access and other research-support tasks, with partial expert baselines.
Scoring and failure modes: Multiple-choice research-support accuracy, with subtask-specific and incomplete human baselines.
System conditions: More than 2,400 tasks; no operational wet-lab execution measurement.
Statistical meaning: Source-reported result statements; this does not establish trial counts, denominators or an estimator: Expert human coverage was incomplete because recruiting qualified baseliners for the entire set was infeasible (§3.5).
Validity and interpretation: The authors do not claim comprehensive biology coverage; unassisted multiple-choice results do not establish real laboratory execution or uplift.
Appropriate research use: Model recommendation: retain subtask, tools and actual human coverage, without extracting operational biological content.

LAB-Bench · AbstractSelected original resource https://arxiv.org/html/2407.10362; sha256:fb4550ba8cc17a4ffb2f30a3e9f1fe78a538a5c1d3508550942723e506ffe2c8; archived source capture; exact capture time not independently established
LAB-Bench · §2.1 Dataset description and constructionSelected original resource https://arxiv.org/html/2407.10362; sha256:fb4550ba8cc17a4ffb2f30a3e9f1fe78a538a5c1d3508550942723e506ffe2c8; archived source capture; exact capture time not independently established
LAB-Bench · §3 Discussion and LimitationsSelected original resource https://arxiv.org/html/2407.10362; sha256:fb4550ba8cc17a4ffb2f30a3e9f1fe78a538a5c1d3508550942723e506ffe2c8; archived source capture; exact capture time not independently established
LAB-Bench · §3.5 Limitations in human evaluationSelected original resource https://arxiv.org/html/2407.10362; sha256:fb4550ba8cc17a4ffb2f30a3e9f1fe78a538a5c1d3508550942723e506ffe2c8; archived source capture; exact capture time not independently established

Evidence across reports

The same benchmark name can cover different tasks, scoring rules and system configurations. Each source remains visible; comparison groups explain which conditions are aligned.

Neo Research

Evaluating Kimi K3 for Misalignment Risks →

Actual evaluators: Neo Research

4 result records · 4 reported system names

Protocols and comparison decisions (0)

Original source charts and materials

LAB-Bench with figures/tables withheld

Original evaluator results · source order

FigQA attempt, figure withheld

0–100%

Kimi K3

Kimi K3 · 79.6% · Evaluates a range of biology research skills with structured questions; text-only or missing-input subsets remain separate protocols. · Figure/table subsets with explicit insufficient-information answer option.

79.6%

GPT-5.6 Terra

GPT-5.6 Terra · 72.9% · Evaluates a range of biology research skills with structured questions; text-only or missing-input subsets remain separate protocols. · Figure/table subsets with explicit insufficient-information answer option.

72.9%

Claude Sonnet 5

Claude Sonnet 5 · 62.4% · Evaluates a range of biology research skills with structured questions; text-only or missing-input subsets remain separate protocols. · Figure/table subsets with explicit insufficient-information answer option.

62.4%

DeepSeek V4 Flash 0731

DeepSeek V4 Flash 0731 · 50.3% · Evaluates a range of biology research skills with structured questions; text-only or missing-input subsets remain separate protocols. · Figure/table subsets with explicit insufficient-information answer option.

50.3%

Showing 4 of 4 systems

Evaluates a range of biology research skills with structured questions; text-only or missing-input subsets remain separate protocols.

These are general scientific capabilities, not end-to-end misuse tasks or direct harm probabilities. Report dates and original executor are shown with this assessment.

Evaluation conditions and system configurations

Kimi K3: Figure/table subsets with explicit insufficient-information answer option. · LAB-Bench with figures/tables withheld; modelVersion: not specified · harness: not specified · tools: not specified · budget: not specified

GPT-5.6 Terra: Figure/table subsets with explicit insufficient-information answer option. · LAB-Bench with figures/tables withheld; modelVersion: not specified · harness: not specified · tools: not specified · budget: not specified

Claude Sonnet 5: Figure/table subsets with explicit insufficient-information answer option. · LAB-Bench with figures/tables withheld; modelVersion: not specified · harness: not specified · tools: not specified · budget: not specified

DeepSeek V4 Flash 0731: Figure/table subsets with explicit insufficient-information answer option. · LAB-Bench with figures/tables withheld; modelVersion: not specified · harness: not specified · tools: not specified · budget: not specified

LAB-Bench with figures/tables withheld · source order
System / configuration	FigQA attempt, figure withheld	TableQA attempt, table withheld	Publication / verification
Kimi K3Harness not specified	79.6%	68.4%	Published2026-09-11
GPT-5.6 TerraHarness not specified	72.9%	61.5%	Published2026-09-11
Claude Sonnet 5Harness not specified	62.4%	53.7%	Published2026-09-11
DeepSeek V4 Flash 0731Harness not specified	50.3%	43.9%	Published2026-09-11

Assessment provenance

Evaluating Kimi K3 for Misalignment Risks → · External report

References

[1] LAB-Bench. 2024-07-14
[2] GLM-5.2 Risk Evaluation Report. 2026-08-02. v1.1 · revised 26 August 2026 ; Report masthead and stated evaluation methodology
[3] Evaluating Kimi K3 for Misalignment Risks. 2026-09-11. September 2026 ; §1.1.2; Figure 2

Search NUWA Index