{"schemaVersion":"nuwa-current-record-1","version":"2026-10-29.1","publication":"published","work":{"id":"vct-v2","title":"Introducing VCT-v2 — the updated Virology Capabilities Test","aliases":["Virology Capabilities Test v2"],"authors":[],"primaryFunction":"evaluation","secondaryFunctions":[],"kind":"benchmark","sourceId":"vct-v2","attributionSourceId":"vct-v2","originalUrl":"https://securebio.org/blog/introducing-vct-v2/","publicationDate":"2026-09-11","updatedOn":"2026-10-25","areas":["cbrn"],"scopeNote":{"en":"Primary dimension is question-answering capability; version and answer-key validity remain material.","zh":"主维度是问答能力；版本与答案效度是重要限制。"},"protocolIds":["vct-v2"],"contributions":[{"organizationId":"securebio","role":"publisher","sourceId":"vct-v2","locator":"Official publication masthead and article byline"}],"summary":{"en":"Revises VCT questions to reduce shortcut-based success and preserve measurement validity.","zh":"修订 VCT 题目以减少捷径作答并提高测量效度。"},"note":{"question":{"en":"Question-level performance in a versioned multimodal test.","zh":"有版本的多模态题目测试表现。"},"method":{"en":"Models answering specialist virology troubleshooting questions","zh":"回答专门病毒学排障问题的模型"},"finding":{"en":"Revises VCT questions to reduce shortcut-based success and preserve measurement validity.","zh":"修订 VCT 题目以减少捷径作答并提高测量效度。"},"supports":{"en":"Primary dimension is question-answering capability; version and answer-key validity remain material.","zh":"主维度是问答能力；版本与答案效度是重要限制。"},"limits":{"en":"Knowledge-question accuracy does not establish practical harmful-task execution; v1 and v2 scores are not directly interchangeable.","zh":"知识问答准确率不证明实际有害任务执行能力；v1 与 v2 分数不可直接互换。"},"connections":null},"resources":[],"noteStatus":{"question":"limited","method":"limited","finding":"limited","supports":"limited","limits":"limited","connections":"limited"},"noteAvailability":{"en":"Existing source-specific draft sections are retained with limited review status. A complete methods, findings and limitations check remains pending; generic sections were omitted.","zh":"已有的来源相关草稿章节予以保留，审核状态为有限核查。完整的方法、发现与限制核查仍待完成；通用套话章节已省略。"},"noteSourceLocator":"Retained existing source-specific note sections; this pass audits duplicate/boilerplate structure and source availability, not complete section-level verification.","tags":[]},"reading":{"id":"vct-v2","title":"Introducing VCT-v2 — the updated Virology Capabilities Test","en":{"question":"Can revisions improve VCT’s answer-key reliability and resistance to shortcuts?","design":"Deterministic response-pattern criteria prioritize expert re-review; the revision removes or corrects problematic items and tests question/image omission.","findings":["VCT-v2 has 279 questions; the top reported model scored 49.2%.","A cross-model per-question aggregate gives a lower bound of 79% on the effective ceiling, not a single-model score."],"interpretation":"NUWA interpretation: revised content and headroom prevent treating VCT and VCT-v2 scores as the same scale.","limits":"The ceiling is only a lower bound; performance on changed questions does not directly validate practical user uplift.","decisionRelevance":"Model recommendation: cite version and question population, preserving the distinction between observed accuracy and inferred headroom.","takeaway":"NUWA interpretation: revised content and headroom prevent treating VCT and VCT-v2 scores as the same scale.","connections":[],"scoring":"Revised question accuracy; cross-model per-question ceiling lower bound is not single-model accuracy.","conditions":"279 questions; content revisions and shortcut tests prevent direct old/new score pooling.","statistics":"VCT-v2 has 279 questions; the top reported model scored 49.2%. A cross-model per-question aggregate gives a lower bound of 79% on the effective ceiling, not a single-model score.","usage":"Model recommendation: cite version and question population, preserving the distinction between observed accuracy and inferred headroom."},"zh":{"question":"修订能否提高VCT答案可靠性与抗捷径能力？","design":"以回答模式确定性规则优先复审，并删除或修改问题，测试缺题干或缺图条件。","findings":["VCT-v2有279题，所列最佳模型49.2%。","逐题跨模型聚合得到有效上限至少79%，并非单模型得分。"],"interpretation":"NUWA 解读：内容与动态范围改变后，两个版本不能直接视为同一尺度。","limits":"上限只是下界；修改题目的表现不能直接验证实际用户增益。","decisionRelevance":"模型建议：引用版本和题目总体，区分观测正确率与推断空间。","takeaway":"NUWA 解读：内容与动态范围改变后，两个版本不能直接视为同一尺度。","connections":[],"scoring":"修订题正确率，跨模型逐题上限下界不是单模型正确率。","conditions":"279题，内容修订及捷径测试使新旧分不能直接汇总。","statistics":"VCT-v2有279题，所列最佳模型49.2%。 逐题跨模型聚合得到有效上限至少79%，并非单模型得分。","usage":"模型建议：引用版本和题目总体，区分观测正确率与推断空间。"},"assertionPolicy":{"question":"source-research-question","design":"source-finding","findings":"source-finding","interpretation":"nuwa-interpretation","limits":"source-limit-or-explicit-editorial-scope-limit","connections":"source-supported-relationship-only","decisionRelevance":"model-recommendation-not-source-conclusion"},"sourceRefs":[{"sourceId":"vct-v2","url":"https://securebio.org/blog/introducing-vct-v2/","locator":"Patching potential issues in VCT","version":"Selected original resource https://securebio.org/blog/introducing-vct-v2/; sha256:be1ae6cf6276b2e9dcd4fd7cb96e652548b619e15cb341e7392245dfcdaa729b; archived source capture; exact capture time not independently established","artifactSha256":"be1ae6cf6276b2e9dcd4fd7cb96e652548b619e15cb341e7392245dfcdaa729b"},{"sourceId":"vct-v2","url":"https://securebio.org/blog/introducing-vct-v2/","locator":"Presenting our updated benchmark — VCT-v2","version":"Selected original resource https://securebio.org/blog/introducing-vct-v2/; sha256:be1ae6cf6276b2e9dcd4fd7cb96e652548b619e15cb341e7392245dfcdaa729b; archived source capture; exact capture time not independently established","artifactSha256":"be1ae6cf6276b2e9dcd4fd7cb96e652548b619e15cb341e7392245dfcdaa729b"},{"sourceId":"vct-v2","url":"https://securebio.org/blog/introducing-vct-v2/","locator":"VCT-v2 is substantially more shortcut-resistant than VCT","version":"Selected original resource https://securebio.org/blog/introducing-vct-v2/; sha256:be1ae6cf6276b2e9dcd4fd7cb96e652548b619e15cb341e7392245dfcdaa729b; archived source capture; exact capture time not independently established","artifactSha256":"be1ae6cf6276b2e9dcd4fd7cb96e652548b619e15cb341e7392245dfcdaa729b"},{"sourceId":"vct-v2","url":"https://securebio.org/blog/introducing-vct-v2/","locator":"Monitoring VCT-v2 for saturation","version":"Selected original resource https://securebio.org/blog/introducing-vct-v2/; sha256:be1ae6cf6276b2e9dcd4fd7cb96e652548b619e15cb341e7392245dfcdaa729b; archived source capture; exact capture time not independently established","artifactSha256":"be1ae6cf6276b2e9dcd4fd7cb96e652548b619e15cb341e7392245dfcdaa729b"}],"publication":{"state":"published","publishedOn":"2026-10-29","authorizationReference":"user-authorized-global-index-publication-2026-10-29","reviewedCandidateId":"local-6efe31bc4a0c0f37"},"review":{"state":"published-note","sourceScope":"full-text","availableScope":null,"readingCoverage":"selected-source-sections; not a complete section-by-section paper audit","completePaperAudit":false,"independentEvidenceReview":"pending-human-review","bilingualEditorialReview":"pending-human-review","authorship":"model-assisted-research-note","preparedFingerprint":"484ae07689df06a9b712d281015d0c670d097d908718b81fe0af4cd23f896551"}},"sources":[{"id":"vct-v2","title":"Introducing VCT-v2 — the updated Virology Capabilities Test","url":"https://securebio.org/blog/introducing-vct-v2/","version":"SecureBio release 2026-09-11","publicationDate":"2026-09-11","datePrecision":"day","type":"benchmark","verifiedOn":"2026-10-25","documentId":"https://securebio.org/blog/introducing-vct-v2"}]}