Risk Evaluation · Benchmark

Virology Capabilities Test (VCT): A Multimodal Virology Q&A Benchmark

NUWA research note · independent review pending

First published: 2025-04-21Source checked: 2026-10-25

Original source ↗Cite this record ↓CBRN →

NUWA Research Note

Research question Design and assumptions Source findings NUWA interpretation Limits and contrary evidence Research and decision relevance Institutions and roles Original resources

Model-assisted reading · Selected sections of the original full text or official article; a complete section-by-section audit remains pending.

At a glance

NUWA interpretation: expert-question performance is not demonstrated wet-lab user uplift.

Research question

How well do models answer expert-authored virology troubleshooting questions?

Design and assumptions

322 multimodal items use all-or-nothing multiple-response scoring; experts are compared within their areas of specialization.

Source findings

o3 scored 43.8%, versus a 22.1% mean for expert virologists with internet access.

The benchmark also analyzes response mismatches and alternative answer formats (§5.1).

NUWA interpretation

NUWA interpretation: expert-question performance is not demonstrated wet-lab user uplift.

Limits and contrary evidence

Experts may disagree about answer keys; embodied and communal tacit knowledge are incompletely represented (§6; Appendix A3).

Research and decision relevance

Model recommendation: retain expertise matching and scoring format; use this only as nonoperational capability evidence.

Precise source locations

Virology Capabilities Test (VCT): A Multimodal Virology Q&A Benchmark · AbstractSelected original resource https://arxiv.org/html/2504.16137v2; sha256:3b0bc61ab556258ca26318138a0ac00dbfb59bb8d3318a8038b71a6788be0d99; archived source capture; exact capture time not independently established
Virology Capabilities Test (VCT): A Multimodal Virology Q&A Benchmark · §5.1 SetupSelected original resource https://arxiv.org/html/2504.16137v2; sha256:3b0bc61ab556258ca26318138a0ac00dbfb59bb8d3318a8038b71a6788be0d99; archived source capture; exact capture time not independently established
Virology Capabilities Test (VCT): A Multimodal Virology Q&A Benchmark · §6 DiscussionSelected original resource https://arxiv.org/html/2504.16137v2; sha256:3b0bc61ab556258ca26318138a0ac00dbfb59bb8d3318a8038b71a6788be0d99; archived source capture; exact capture time not independently established
Virology Capabilities Test (VCT): A Multimodal Virology Q&A Benchmark · Appendix A3 Limitations of This WorkSelected original resource https://arxiv.org/html/2504.16137v2; sha256:3b0bc61ab556258ca26318138a0ac00dbfb59bb8d3318a8038b71a6788be0d99; archived source capture; exact capture time not independently established

Source checking, extraction, editorial approval and replication are recorded separately. Methodology →

Institutions and roles

Research by SecureBio Federal University of ABC Center for AI Safety Massachusetts Institute of Technology

Authors (9)

Jasper Götting · Pedro Medeiros · Jon G Sanders · Nathaniel Li · Long Phan · Karam Elabd · Lennart Justen · Dan Hendrycks · Seth Donoughe

Evaluation protocols

Virology Capabilities Test (VCT) →

Cite and reuse

Cite the original publication for its findings. Cite this NUWA record separately for its explanation and retained snapshot.

BibTeX ↓RIS ↓Record + source metadata · JSON ↓Reading coverage · JSON Suggest a correction

Original publication · BibTeX

@misc{vct,
  title = {Virology Capabilities Test (VCT): A Multimodal Virology Q\&A Benchmark},
  url = {https://arxiv.org/abs/2504.16137v2},
  author = {Jasper Götting and Pedro Medeiros and Jon G Sanders and Nathaniel Li and Long Phan and Karam Elabd and Lennart Justen and Dan Hendrycks and Seth Donoughe},
  year = {2025},
  date = {2025-04-21},
  version = {arXiv:2504.16137v2},
  note = {arXiv:2504.16137v2}
}

NUWA record · BibTeX

@misc{vct-nuwa,
  title = {Virology Capabilities Test (VCT): A Multimodal Virology Q\&A Benchmark — NUWA research record},
  url = {https://whitzard.tech/en/nuwa/whitzard-index/work/vct/},
  version = {2026-10-29.1},
  organization = {NUWA Frontier AI Safety Lab},
  howpublished = {Retained note edition: https://whitzard.tech/nuwa/whitzard-index/data/record-editions/2026-10-29.1--vct.json},
  note = {NUWA research note. Source-reading scope and independent review status are stated on the record. Snapshot 2026-10-29.1. Retained note edition: https://whitzard.tech/nuwa/whitzard-index/data/record-editions/2026-10-29.1--vct.json.}
}

Evidence snapshot 2026-10-29.1 · Index updates · RSS

References

[1] Jasper Götting a, Pedro Medeiros a, et al.. Virology Capabilities Test (VCT): A Multimodal Virology Q&A Benchmark. 2025-04-29. v2 [arXiv]; Author affiliations on the original paper title page; Abstract; §5.1 Setup