{"schemaVersion":"nuwa-current-record-1","version":"2026-10-29.1","publication":"published","work":{"id":"biotier","title":"BioTIER: A Refusal Benchmark for Targeted Biological Risk Mitigation","aliases":["BioTIER"],"authors":["Eleanor M. Marshall","Pedro Medeiros","Peter Peneder","Nelly Mak","Jacob Kaffey","Faith Rovenolt","Mac Walker","Seth Donoughe","Jasper Götting"],"primaryFunction":"evaluation","secondaryFunctions":[],"kind":"benchmark","sourceId":"biotier","attributionSourceId":"biotier-author-source","originalUrl":"https://arxiv.org/abs/2607.14479v1","publicationDate":"2026-07-16","updatedOn":"2026-10-25","areas":["cbrn"],"scopeNote":{"en":"Primary observation is what the model outputs or refuses in the test.","zh":"主观测是模型在试验中的输出或拒答。"},"protocolIds":["biotier"],"contributions":[{"organizationId":"securebio","role":"research","sourceId":"biotier-author-source","locator":"Author affiliations on the original paper title page"},{"organizationId":"ufabc","role":"research","sourceId":"biotier-author-source","locator":"Author affiliations on the original paper title page"}],"summary":{"en":"Evaluates refusal of high-risk requests alongside preservation of benign biological assistance.","zh":"同时评测高风险请求拒答与正常生物研究协助的保留。"},"note":{"question":{"en":"Response and refusal behavior across expert-curated prompt sets.","zh":"在专家整理的提示集合中观察响应与拒答行为。"},"method":{"en":"Models responding to tiered biological-risk prompts","zh":"响应分层生物风险提示的模型"},"finding":{"en":"Evaluates refusal of high-risk requests alongside preservation of benign biological assistance.","zh":"同时评测高风险请求拒答与正常生物研究协助的保留。"},"supports":{"en":"Primary observation is what the model outputs or refuses in the test.","zh":"主观测是模型在试验中的输出或拒答。"},"limits":{"en":"Refusal is a safeguard outcome; it does not directly measure underlying biological knowledge or task ability.","zh":"拒答属于防护结果，不直接测量潜在生物知识或任务能力。"},"connections":null},"resources":[],"noteStatus":{"question":"limited","method":"limited","finding":"limited","supports":"limited","limits":"limited","connections":"limited"},"noteAvailability":{"en":"Existing source-specific draft sections are retained with limited review status. A complete methods, findings and limitations check remains pending; generic sections were omitted.","zh":"已有的来源相关草稿章节予以保留，审核状态为有限核查。完整的方法、发现与限制核查仍待完成；通用套话章节已省略。"},"noteSourceLocator":"Retained existing source-specific note sections; this pass audits duplicate/boilerplate structure and source availability, not complete section-level verification.","tags":[]},"reading":{"id":"biotier","title":"BioTIER: A Refusal Benchmark for Targeted Biological Risk Mitigation","en":{"question":"Can biological safeguards distinguish narrowly high-risk requests from benign scientific access?","design":"542 expert-curated prompts span three risk sets; 52 models from ten developers are assessed with binary substantive-response/refusal labels.","findings":["The evaluation explicitly distinguishes refusing restricted content from permitting benign content.","Binary compliance does not score answer correctness, completeness or real-world uplift (§3.3)."],"interpretation":"NUWA interpretation: targeted access-policy compliance is different from biological capability or realized harm.","limits":"Taxonomy thresholds involve expert judgment and need periodic updating; benchmark-aware training remains possible (§5.4).","decisionRelevance":"Model recommendation: report under-refusal and over-refusal separately and avoid interpreting compliance as an operational-harm score.","takeaway":"NUWA interpretation: targeted access-policy compliance is different from biological capability or realized harm.","connections":[],"scoring":"Binary substantive-answer/refusal compliance, with restricted-refuse and benign-permit tasks separated.","conditions":"542 prompts, 52 models; highest publicly available reasoning configurations.","statistics":"Binary compliance does not score answer correctness, completeness or real-world uplift (§3.3).","usage":"Model recommendation: report under-refusal and over-refusal separately and avoid interpreting compliance as an operational-harm score."},"zh":{"question":"生物防护能否区分狭窄高风险请求与良性科研访问？","design":"542项专家提示分三个风险集合；对十家开发者52个模型以实质回答或拒绝作二元判分。","findings":["评价分别检查拒绝受限信息和允许良性信息。","二元遵从不评分正确性、完整性或现实增益（§3.3）。"],"interpretation":"NUWA 解读：定向访问政策遵从不同于生物能力或实际伤害。","limits":"分类阈值含专家判断，须定期更新，且存在针对基准训练的可能（§5.4）。","decisionRelevance":"模型建议：分别报告拒绝不足与过度拒绝，不将遵从视为操作性伤害分数。","takeaway":"NUWA 解读：定向访问政策遵从不同于生物能力或实际伤害。","connections":[],"scoring":"实质回答或拒绝的二元遵从，分受限应拒与良性应答。","conditions":"542提示、52模型，采用公开最高推理配置。","statistics":"二元遵从不评分正确性、完整性或现实增益（§3.3）。","usage":"模型建议：分别报告拒绝不足与过度拒绝，不将遵从视为操作性伤害分数。"},"assertionPolicy":{"question":"source-research-question","design":"source-finding","findings":"source-finding","interpretation":"nuwa-interpretation","limits":"source-limit-or-explicit-editorial-scope-limit","connections":"source-supported-relationship-only","decisionRelevance":"model-recommendation-not-source-conclusion"},"sourceRefs":[{"sourceId":"biotier-author-source","url":"https://arxiv.org/html/2607.14479v1","locator":"Abstract","version":"Selected original resource https://arxiv.org/html/2607.14479v1; sha256:d39bab620389ee6f797ce66762dc0bda4f5afdf89d5653bcbd08a2f0970b9e01; archived source capture; exact capture time not independently established","artifactSha256":"d39bab620389ee6f797ce66762dc0bda4f5afdf89d5653bcbd08a2f0970b9e01"},{"sourceId":"biotier-author-source","url":"https://arxiv.org/html/2607.14479v1","locator":"§3.3 Evaluation","version":"Selected original resource https://arxiv.org/html/2607.14479v1; sha256:d39bab620389ee6f797ce66762dc0bda4f5afdf89d5653bcbd08a2f0970b9e01; archived source capture; exact capture time not independently established","artifactSha256":"d39bab620389ee6f797ce66762dc0bda4f5afdf89d5653bcbd08a2f0970b9e01"},{"sourceId":"biotier-author-source","url":"https://arxiv.org/html/2607.14479v1","locator":"§5.4 Limitations","version":"Selected original resource https://arxiv.org/html/2607.14479v1; sha256:d39bab620389ee6f797ce66762dc0bda4f5afdf89d5653bcbd08a2f0970b9e01; archived source capture; exact capture time not independently established","artifactSha256":"d39bab620389ee6f797ce66762dc0bda4f5afdf89d5653bcbd08a2f0970b9e01"}],"publication":{"state":"published","publishedOn":"2026-10-29","authorizationReference":"user-authorized-global-index-publication-2026-10-29","reviewedCandidateId":"local-6efe31bc4a0c0f37"},"review":{"state":"published-note","sourceScope":"full-text","availableScope":null,"readingCoverage":"selected-source-sections; not a complete section-by-section paper audit","completePaperAudit":false,"independentEvidenceReview":"pending-human-review","bilingualEditorialReview":"pending-human-review","authorship":"model-assisted-research-note","preparedFingerprint":"c61a33da29671b66a498143ce24fa6b244eead4108dfccad8e54d341dc08c108"}},"sources":[{"id":"biotier","title":"BioTIER: A Refusal Benchmark for Targeted Biological Risk Mitigation","url":"https://arxiv.org/abs/2607.14479v1","version":"arXiv:2607.14479v1","publicationDate":"2026-07-16","datePrecision":"day","type":"benchmark","verifiedOn":"2026-10-25","documentId":"arxiv:2607.14479","authors":["Eleanor M. Marshall","Pedro Medeiros","Peter Peneder","Nelly Mak","Jacob Kaffey","Faith Rovenolt","Mac Walker","Seth Donoughe","Jasper Götting"],"aliases":["https://doi.org/10.48550/arXiv.2607.14479","https://arxiv.org/abs/2607.14479","https://arxiv.org/pdf/2607.14479"]}]}