Virology Capabilities Test v2

Source-method reading and retained catalogue information are distinguished. Publication does not establish independent evidence or bilingual editorial review.

Measurement construct

Can revisions improve VCT’s answer-key reliability and resistance to shortcuts?

Task population

Deterministic response-pattern criteria prioritize expert re-review; the revision removes or corrects problematic items and tests question/image omission.

Scoring and failure modes

Revised question accuracy; cross-model per-question ceiling lower bound is not single-model accuracy.

System conditions

279 questions; content revisions and shortcut tests prevent direct old/new score pooling.

Statistical meaning

Source-reported result statements; this does not establish trial counts, denominators or an estimator: VCT-v2 has 279 questions; the top reported model scored 49.2%. A cross-model per-question aggregate gives a lower bound of 79% on the effective ceiling, not a single-model score.

Validity and interpretation

The ceiling is only a lower bound; performance on changed questions does not directly validate practical user uplift.

Appropriate research use

Model recommendation: cite version and question population, preserving the distinction between observed accuracy and inferred headroom.

• Introducing VCT-v2 — the updated Virology Capabilities Test · Patching potential issues in VCTSelected original resource https://securebio.org/blog/introducing-vct-v2/; sha256:be1ae6cf6276b2e9dcd4fd7cb96e652548b619e15cb341e7392245dfcdaa729b; archived source capture; exact capture time not independently established
• Introducing VCT-v2 — the updated Virology Capabilities Test · Presenting our updated benchmark — VCT-v2Selected original resource https://securebio.org/blog/introducing-vct-v2/; sha256:be1ae6cf6276b2e9dcd4fd7cb96e652548b619e15cb341e7392245dfcdaa729b; archived source capture; exact capture time not independently established
• Introducing VCT-v2 — the updated Virology Capabilities Test · VCT-v2 is substantially more shortcut-resistant than VCTSelected original resource https://securebio.org/blog/introducing-vct-v2/; sha256:be1ae6cf6276b2e9dcd4fd7cb96e652548b619e15cb341e7392245dfcdaa729b; archived source capture; exact capture time not independently established
• Introducing VCT-v2 — the updated Virology Capabilities Test · Monitoring VCT-v2 for saturationSelected original resource https://securebio.org/blog/introducing-vct-v2/; sha256:be1ae6cf6276b2e9dcd4fd7cb96e652548b619e15cb341e7392245dfcdaa729b; archived source capture; exact capture time not independently established