NUWA 风险评测 · 数据集
MALT: A Dataset of Natural and Prompted Behaviors That Threaten Eval Integrity
NUWA 研究解读 · 独立复核待完成
NUWA Research Note
研究问题研究设计与假设来源发现NUWA 解读局限与相反证据研究与决策意义机构与角色原始资源模型辅助整理 · 选读原始全文或官方文章中的相关章节；尚未完成逐章节审计。
研究速读
NUWA 解读：有标签检测数据不能当无偏发生率样本。
研究问题
人工标注轨迹能否支撑评测完整性故障检测？
研究设计与假设
MALT收集21模型、403任务的10,919条轨迹，混合自然与提示行为，公开集和复审集不同。
来源发现
人工复审2,690条，其中良性1,014条、有相关行为1,676条。
另8,229条未充分检查，监测结果仅用复审集。
NUWA 解读
NUWA 解读：有标签检测数据不能当无偏发生率样本。
局限与相反证据
收集含异常筛查和提示诱发，未检查集估计标签错误6%，公开集也按任务筛选。
研究与决策意义
模型建议：监测结论中分开复审、公开及提示或自发总体。
精确来源位置
- MALT: A Dataset of Natural and Prompted Behaviors That Threaten Eval Integrity · SummarySelected original resource https://metr.org/blog/2025-10-14-malt-dataset-of-natural-and-prompted-behaviors/; sha256:07cbb96eaeb01abc8876dd62deac9039e258b1ddb0d4944e507b830e09148c5f; archived source capture; exact capture time not independently established
- MALT: A Dataset of Natural and Prompted Behaviors That Threaten Eval Integrity · Dataset detailsSelected original resource https://metr.org/blog/2025-10-14-malt-dataset-of-natural-and-prompted-behaviors/; sha256:07cbb96eaeb01abc8876dd62deac9039e258b1ddb0d4944e507b830e09148c5f; archived source capture; exact capture time not independently established
- MALT: A Dataset of Natural and Prompted Behaviors That Threaten Eval Integrity · Dataset collectionSelected original resource https://metr.org/blog/2025-10-14-malt-dataset-of-natural-and-prompted-behaviors/; sha256:07cbb96eaeb01abc8876dd62deac9039e258b1ddb0d4944e507b830e09148c5f; archived source capture; exact capture time not independently established
- MALT: A Dataset of Natural and Prompted Behaviors That Threaten Eval Integrity · Monitoring Setup and ResultsSelected original resource https://metr.org/blog/2025-10-14-malt-dataset-of-natural-and-prompted-behaviors/; sha256:07cbb96eaeb01abc8876dd62deac9039e258b1ddb0d4944e507b830e09148c5f; archived source capture; exact capture time not independently established
- MALT: A Dataset of Natural and Prompted Behaviors That Threaten Eval Integrity · LimitationsSelected original resource https://metr.org/blog/2025-10-14-malt-dataset-of-natural-and-prompted-behaviors/; sha256:07cbb96eaeb01abc8876dd62deac9039e258b1ddb0d4944e507b830e09148c5f; archived source capture; exact capture time not independently established
来源核查、数据提取、编辑批准与实验复现分别记录。 方法说明 →
机构与角色
引用与复用
引用研究发现时请引用原始出版物；引用 NUWA 的整理与解释时，单独引用本站记录及对应快照。
原始出版物 · BibTeX
@misc{malt,
title = {MALT: A Dataset of Natural and Prompted Behaviors That Threaten Eval Integrity},
url = {https://metr.org/blog/2025-10-14-malt-dataset-of-natural-and-prompted-behaviors/},
year = {2025},
date = {2025-10-14},
version = {Source capture 2026-10-25},
note = {Source capture 2026-10-25}
}
NUWA 记录 · BibTeX
@misc{malt-nuwa,
title = {MALT: A Dataset of Natural and Prompted Behaviors That Threaten Eval Integrity — NUWA research record},
url = {https://whitzard.tech/nuwa/whitzard-index/work/malt/},
version = {2026-10-29.1},
organization = {NUWA Frontier AI Safety Lab},
howpublished = {Retained note edition: https://whitzard.tech/nuwa/whitzard-index/data/record-editions/2026-10-29.1--malt.json},
note = {NUWA research note. Source-reading scope and independent review status are stated on the record. Snapshot 2026-10-29.1. Retained note edition: https://whitzard.tech/nuwa/whitzard-index/data/record-editions/2026-10-29.1--malt.json.}
}
证据快照 2026-10-29.1 · 索引更新 · RSS
参考文献
- [1] MALT: A Dataset of Natural and Prompted Behaviors That Threaten Eval Integrity. 2025-10-14 ; Official publication masthead and article byline; Summary; Dataset details