风险评测 · 评测基准

Evaluating Claude’s bioinformatics research capabilities with BioMysteryBench

NUWA 研究解读 · 独立复核待完成

首次发表: 发表日期尚未核实来源核验: 2026-10-25

原始来源 ↗引用本记录 ↓化生放核 →

NUWA Research Note

研究问题研究设计与假设来源发现 NUWA 解读局限与相反证据研究与决策意义机构与角色原始资源

模型辅助整理 · 选读原始全文或官方文章中的相关章节；尚未完成逐章节审计。

研究速读

NUWA 解读：人类困难只指所招募尝试中未解，并非人类不可能。

研究问题

生信智能体能否解决不同于考试题的可验证研究问题？

研究设计与假设

任务有可核答案但允许不同路径，每题最多五位专家尝试，模型按每题五次取平均。

来源发现

76题至少一位人类解决。

删除四道缺陷题后，23题仍为人类困难，Mythos Preview解其中约30%。

NUWA 解读

NUWA 解读：人类困难只指所招募尝试中未解，并非人类不可能。

局限与相反证据

小型筛选集、专家招募和删题限制超人类结论，未测湿实验增益。

研究与决策意义

模型建议：保留子集定义与五次平均，省略操作性生物题目细节。

精确来源位置

BioMysteryBench · Benchmarking models on verifiable biological tasks with BioMysteryBenchSelected original resource https://www.anthropic.com/research/Evaluating-Claude-For-Bioinformatics-With-BioMysteryBench; sha256:669c8f9a60c28899106a0784a0aea320de59a48eed1369395bedd3ec6fa71cc3; archived source capture; exact capture time not independently established
BioMysteryBench · Human baselining: Human-solvableSelected original resource https://www.anthropic.com/research/Evaluating-Claude-For-Bioinformatics-With-BioMysteryBench; sha256:669c8f9a60c28899106a0784a0aea320de59a48eed1369395bedd3ec6fa71cc3; archived source capture; exact capture time not independently established
BioMysteryBench · Human baselining: Human-difficultSelected original resource https://www.anthropic.com/research/Evaluating-Claude-For-Bioinformatics-With-BioMysteryBench; sha256:669c8f9a60c28899106a0784a0aea320de59a48eed1369395bedd3ec6fa71cc3; archived source capture; exact capture time not independently established

来源核查、数据提取、编辑批准与实验复现分别记录。方法说明 →

机构与角色

原文未注明机构

评测协议

BioMysteryBench →

引用与复用

引用研究发现时请引用原始出版物；引用 NUWA 的整理与解释时，单独引用本站记录及对应快照。

BibTeX ↓RIS ↓记录与来源元数据 · JSON ↓阅读覆盖 · JSON 提交更正

原始出版物 · BibTeX

@misc{biomysterybench,
  title = {Evaluating Claude’s bioinformatics research capabilities with BioMysteryBench},
  url = {https://www.anthropic.com/research/Evaluating-Claude-For-Bioinformatics-With-BioMysteryBench},
  version = {2026-10-25 source capture},
  note = {2026-10-25 source capture}
}

NUWA 记录 · BibTeX

@misc{biomysterybench-nuwa,
  title = {Evaluating Claude’s bioinformatics research capabilities with BioMysteryBench — NUWA research record},
  url = {https://whitzard.tech/nuwa/whitzard-index/work/biomysterybench/},
  version = {2026-10-29.1},
  organization = {NUWA Frontier AI Safety Lab},
  howpublished = {Retained note edition: https://whitzard.tech/nuwa/whitzard-index/data/record-editions/2026-10-29.1--biomysterybench.json},
  note = {NUWA research note. Source-reading scope and independent review status are stated on the record. Snapshot 2026-10-29.1. Retained note edition: https://whitzard.tech/nuwa/whitzard-index/data/record-editions/2026-10-29.1--biomysterybench.json.}
}

证据快照 2026-10-29.1 · 索引更新 · RSS

参考文献

[1] BioMysteryBench ; Benchmarking models on verifiable biological tasks with BioMysteryBench; Human baselining: Human-solvable; Human baselining: Human-difficult