Risk Evaluation · Benchmark

Evaluating Claude’s bioinformatics research capabilities with BioMysteryBench

NUWA research note · independent review pending

First published: Publication date not verifiedSource checked: 2026-10-25

Original source ↗Cite this record ↓CBRN →

NUWA Research Note

Research question Design and assumptions Source findings NUWA interpretation Limits and contrary evidence Research and decision relevance Institutions and roles Original resources

Model-assisted reading · Selected sections of the original full text or official article; a complete section-by-section audit remains pending.

At a glance

NUWA interpretation: “human-difficult” means unsolved within this recruited attempt process, not impossible for humans.

Research question

Can bioinformatics agents solve verifiable research questions that differ from exam-style biology?

Design and assumptions

Tasks have checkable answers but allow varied solution routes; up to five experts attempt each question, and model accuracy averages five trials per problem.

Source findings

76 tasks were solved by at least one human.

After removing four flawed questions, 23 remained human-difficult; Mythos Preview solved about 30% of that subset.

NUWA interpretation

NUWA interpretation: “human-difficult” means unsolved within this recruited attempt process, not impossible for humans.

Limits and contrary evidence

Small selected subsets, expert recruitment and removed flawed questions constrain claims of human superiority; no wet-lab uplift is measured.

Research and decision relevance

Model recommendation: retain human-solvable/difficult definitions and five-trial averaging; omit operational biological task details.

Precise source locations

BioMysteryBench · Benchmarking models on verifiable biological tasks with BioMysteryBenchSelected original resource https://www.anthropic.com/research/Evaluating-Claude-For-Bioinformatics-With-BioMysteryBench; sha256:669c8f9a60c28899106a0784a0aea320de59a48eed1369395bedd3ec6fa71cc3; archived source capture; exact capture time not independently established
BioMysteryBench · Human baselining: Human-solvableSelected original resource https://www.anthropic.com/research/Evaluating-Claude-For-Bioinformatics-With-BioMysteryBench; sha256:669c8f9a60c28899106a0784a0aea320de59a48eed1369395bedd3ec6fa71cc3; archived source capture; exact capture time not independently established
BioMysteryBench · Human baselining: Human-difficultSelected original resource https://www.anthropic.com/research/Evaluating-Claude-For-Bioinformatics-With-BioMysteryBench; sha256:669c8f9a60c28899106a0784a0aea320de59a48eed1369395bedd3ec6fa71cc3; archived source capture; exact capture time not independently established

Source checking, extraction, editorial approval and replication are recorded separately. Methodology →

Institutions and roles

Affiliation not specified

Evaluation protocols

BioMysteryBench →

Cite and reuse

Cite the original publication for its findings. Cite this NUWA record separately for its explanation and retained snapshot.

BibTeX ↓RIS ↓Record + source metadata · JSON ↓Reading coverage · JSON Suggest a correction

Original publication · BibTeX

@misc{biomysterybench,
  title = {Evaluating Claude’s bioinformatics research capabilities with BioMysteryBench},
  url = {https://www.anthropic.com/research/Evaluating-Claude-For-Bioinformatics-With-BioMysteryBench},
  version = {2026-10-25 source capture},
  note = {2026-10-25 source capture}
}

NUWA record · BibTeX

@misc{biomysterybench-nuwa,
  title = {Evaluating Claude’s bioinformatics research capabilities with BioMysteryBench — NUWA research record},
  url = {https://whitzard.tech/en/nuwa/whitzard-index/work/biomysterybench/},
  version = {2026-10-29.1},
  organization = {NUWA Frontier AI Safety Lab},
  howpublished = {Retained note edition: https://whitzard.tech/nuwa/whitzard-index/data/record-editions/2026-10-29.1--biomysterybench.json},
  note = {NUWA research note. Source-reading scope and independent review status are stated on the record. Snapshot 2026-10-29.1. Retained note edition: https://whitzard.tech/nuwa/whitzard-index/data/record-editions/2026-10-29.1--biomysterybench.json.}
}

Evidence snapshot 2026-10-29.1 · Index updates · RSS

References

[1] BioMysteryBench ; Benchmarking models on verifiable biological tasks with BioMysteryBench; Human baselining: Human-solvable; Human baselining: Human-difficult