Evidence record
o3 (Apr ’25)
Virology Capabilities Test (VCT) · Evidence record
Original source charts and materials
Reported results
Source order · reported configurations
Select systems
Multiple-response accuracy
0–100%Values describe the measured outcome under the source protocol, not a general safety score.
Table 1: multiple-response accuracy. Three evaluation runs, except italicized source entries evaluated once; the asterisk denotes up to 4096 reasoning tokens. Knowledge accuracy is not end-to-end harmful capability. Evaluation date not specified; publication / record dates are shown below.
Evaluation conditions and system configurations
- o3 (Apr ’25)
- Three evaluation runs; source reports SEM below 1% for all models. · Table 1 · Virology Capabilities Test (VCT)
- modelVersion: not specified · harness: not specified · tools: not specified · budget: not specified
| System / configuration | Multiple-response accuracy | Publication / verification |
|---|---|---|
| 43.8% | Source checked2026-10-25 |
Method and scope
Question-level performance in a versioned multimodal test.
- Evidence type
- Benchmark
- Assessment subject
- Models answering specialist virology troubleshooting questions
- Primary dimension
- Capability
- Evidence snapshot
- 2026-10-29.1
Primary dimension is question-answering capability; version and answer-key validity remain material.
Source finding
Source-reported measurements for o3 (Apr ’25) in Table 1; values are transcribed, not independently reproduced.
Limitations
Knowledge-question accuracy does not establish practical harmful-task execution; v1 and v2 scores are not directly interchangeable.
Configuration, task and aggregation differences remain as reported. Source order is retained. Transcription is not independent replication; no cross-protocol ranking is supported.
Related research questions
CBRN / Hazardous knowledge →
The source tests domain knowledge and troubleshooting as a proxy, not end-to-end task execution.
References
Numerical results are reported by the cited authors.
- [1] Jasper Götting a, Pedro Medeiros a, et al.. Virology Capabilities Test (VCT): A Multimodal Virology Q&A Benchmark. 2025-04-29. v2 [arXiv]; Author affiliations on the original paper title page; Table 1