NUWA 风险评测 · 研究
Auditing FORTRESS: A Multi-Judge Panel Doesn’t Guarantee Robustness
NUWA 研究解读 · 独立复核待完成
NUWA Research Note
研究问题研究设计与假设来源发现NUWA 解读局限与相反证据研究联系研究与决策意义机构与角色原始资源模型辅助整理 · 选读原始全文或官方文章中的相关章节；尚未完成逐章节审计。
研究速读
NUWA 解读：多判官不保证独立或有效测量。
研究问题
多个模型判官仍会系统性误判防护回答吗？
研究设计与假设
Generality审计FORTRESS的Inspect数据、求解器、计分和投票，区分量表满足与真实回答内容。
来源发现
重判使ARS最高降35%、ORS最高降88%。
判官有时对攻击请求而非空回答判分，投票规则又放大错误。
NUWA 解读
NUWA 解读：多判官不保证独立或有效测量。
局限与相反证据
审计针对特定实现及重判政策，降分不是模型防护本身发生变化。
研究联系
本文审计FORTRESS计分实现，是方法跟进，不是独立基准总体。
研究与决策意义
模型建议：使用多判官综合分前测试空回答、弃权及投票逻辑。
精确来源位置
- Auditing FORTRESS: A Multi-Judge Panel Doesn’t Guarantee Robustness · Eval architecture overviewSelected original resource https://generality.org/blog/posts/fortress-fragile-judges/; sha256:ac0c18b1ee9c3d14472e55bf6a660ab6f53f21cba2eab8ef6b7e254139c97a4d; archived source capture; exact capture time not independently established
- Auditing FORTRESS: A Multi-Judge Panel Doesn’t Guarantee Robustness · FORTRESS’s Fragile Judging PanelSelected original resource https://generality.org/blog/posts/fortress-fragile-judges/; sha256:ac0c18b1ee9c3d14472e55bf6a660ab6f53f21cba2eab8ef6b7e254139c97a4d; archived source capture; exact capture time not independently established
- Auditing FORTRESS: A Multi-Judge Panel Doesn’t Guarantee Robustness · What does FORTRESS actually measure?Selected original resource https://generality.org/blog/posts/fortress-fragile-judges/; sha256:ac0c18b1ee9c3d14472e55bf6a660ab6f53f21cba2eab8ef6b7e254139c97a4d; archived source capture; exact capture time not independently established
- Auditing FORTRESS: A Multi-Judge Panel Doesn’t Guarantee Robustness · Constructing stronger FORTRESSesSelected original resource https://generality.org/blog/posts/fortress-fragile-judges/; sha256:ac0c18b1ee9c3d14472e55bf6a660ab6f53f21cba2eab8ef6b7e254139c97a4d; archived source capture; exact capture time not independently established
来源核查、数据提取、编辑批准与实验复现分别记录。 方法说明 →
机构与角色
作者 (1)
Laurence Wroe
关联研究
FORTRESS: Frontier Risk Evaluation for National Security and Public Safety
审计 FORTRESS 的评分过程如何影响报告分数。
引用与复用
引用研究发现时请引用原始出版物；引用 NUWA 的整理与解释时，单独引用本站记录及对应快照。
原始出版物 · BibTeX
@misc{fortress-audit,
title = {Auditing FORTRESS: A Multi-Judge Panel Doesn’t Guarantee Robustness},
url = {https://generality.org/blog/posts/fortress-fragile-judges/},
author = {Laurence Wroe},
version = {Source capture 2026-10-25},
note = {Source capture 2026-10-25}
}
NUWA 记录 · BibTeX
@misc{fortress-audit-nuwa,
title = {Auditing FORTRESS: A Multi-Judge Panel Doesn’t Guarantee Robustness — NUWA research record},
url = {https://whitzard.tech/nuwa/whitzard-index/work/fortress-audit/},
version = {2026-10-29.1},
organization = {NUWA Frontier AI Safety Lab},
howpublished = {Retained note edition: https://whitzard.tech/nuwa/whitzard-index/data/record-editions/2026-10-29.1--fortress-audit.json},
note = {NUWA research note. Source-reading scope and independent review status are stated on the record. Snapshot 2026-10-29.1. Retained note edition: https://whitzard.tech/nuwa/whitzard-index/data/record-editions/2026-10-29.1--fortress-audit.json.}
}
证据快照 2026-10-29.1 · 索引更新 · RSS
参考文献
- [1] Laurence Wroe. Auditing FORTRESS: A Multi-Judge Panel Doesn’t Guarantee Robustness ; Article byline, 18 September 2026; What is FORTRESS?; audit methodology; Research background and discussion in the original publication; Eval architecture overview