NUWA 评测档案
TraceProbe
155 场景、12 模型、每设置 3 次运行；分别报告任务成功、终态违规、攻击子集成功和潜在中间违规。
证据补充 · 2026-10-28
模型辅助整理 · 独立证据及双语编辑复核待完成。以下数值为来源发现或材料计数，与趋势结果序列分开保存。
一项来源描述的沙箱案例
论文的争议处理示例中，智能体完成有用摘要，却在过度信任内部消息后读取受保护的模拟数据并发往模拟接收端。这是构造沙箱案例，不是现实事故。
示例指标为 TSR=1、SVR=1、ASR=1、潜在违规=1、STCR=0；任务成功与中间、终点违规并存。
案例解释指标之间的区别，不代表总体发生率或干预因果效果。既定动作回放能够定位越界位置，不重新运行模型决策。
原始案例 ↗ · §3.2 illustrative case-dispute scenario; Table 24
原文数值与条件（69 条）
计数、比例与对比效应使用不同单位。样本数量不等于独立实验数量；未披露的分子和区间水平保持未知。
| 系统／指标 | 原文数值 | 总体与条件 |
|---|---|---|
| Claude Opus 4.6 TSR | 0.852 比例 | 分母: 155 场景 · scenario mean after averaging 3 repeats 每场景重复: 3 · 执行: 465 (NUWA 计算) 条件、限制与来源{
"condition": "standard prompting",
"temperature": "0.0 where exposed, otherwise provider default",
"tokenCapOverride": null,
"seed": null
}Constructed scenario distribution, not natural prevalence; no raw numerator inferred from rounded rate. 原文 ↗ · Table2 PDFp7; §3.6; Table9; text lines 00623,01737Original capture 2026-10-28T07:24:41.534922+00:00; sha256:b0b85491f4be9fb842c2f97891b033a75b44a85c756b0f3b92cf8da775d8f828 |
| Claude Opus 4.6 SVR | 0.103 比例 | 分母: 155 场景 · scenario mean after averaging 3 repeats 每场景重复: 3 · 执行: 465 (NUWA 计算) 条件、限制与来源{
"condition": "standard prompting",
"temperature": "0.0 where exposed, otherwise provider default",
"tokenCapOverride": null,
"seed": null
}Constructed scenario distribution, not natural prevalence; no raw numerator inferred from rounded rate. 原文 ↗ · Table2 PDFp7; §3.6; Table9; text lines 00623,01737Original capture 2026-10-28T07:24:41.534922+00:00; sha256:b0b85491f4be9fb842c2f97891b033a75b44a85c756b0f3b92cf8da775d8f828 |
| Claude Opus 4.6 ASR | 0.095 比例 | 分母: 105 攻击标注场景 · scenario mean after averaging 3 repeats 每场景重复: 3 · 执行: 315 (NUWA 计算) 条件、限制与来源{
"condition": "standard prompting",
"temperature": "0.0 where exposed, otherwise provider default",
"tokenCapOverride": null,
"seed": null
}Constructed scenario distribution, not natural prevalence; no raw numerator inferred from rounded rate. 原文 ↗ · Table2 PDFp7; §3.6; Table9; text lines 00623,01737Original capture 2026-10-28T07:24:41.534922+00:00; sha256:b0b85491f4be9fb842c2f97891b033a75b44a85c756b0f3b92cf8da775d8f828 |
| Claude Opus 4.6 STCR | 0.755 比例 | 分母: 155 场景 · scenario mean after averaging 3 repeats 每场景重复: 3 · 执行: 465 (NUWA 计算) 条件、限制与来源{
"condition": "standard prompting",
"temperature": "0.0 where exposed, otherwise provider default",
"tokenCapOverride": null,
"seed": null
}Constructed scenario distribution, not natural prevalence; no raw numerator inferred from rounded rate. 原文 ↗ · Table2 PDFp7; §3.6; Table9; text lines 00623,01737Original capture 2026-10-28T07:24:41.534922+00:00; sha256:b0b85491f4be9fb842c2f97891b033a75b44a85c756b0f3b92cf8da775d8f828 |
| Claude Opus 4.6 Latent | 0.097 比例 | 分母: 155 场景 · scenario mean after averaging 3 repeats 每场景重复: 3 · 执行: 465 (NUWA 计算) 条件、限制与来源{
"condition": "standard prompting",
"temperature": "0.0 where exposed, otherwise provider default",
"tokenCapOverride": null,
"seed": null
}Constructed scenario distribution, not natural prevalence; no raw numerator inferred from rounded rate. 原文 ↗ · Table2 PDFp7; §3.6; Table9; text lines 00623,01737Original capture 2026-10-28T07:24:41.534922+00:00; sha256:b0b85491f4be9fb842c2f97891b033a75b44a85c756b0f3b92cf8da775d8f828 |
| Claude Sonnet 4.6 TSR | 0.819 比例 | 分母: 155 场景 · scenario mean after averaging 3 repeats 每场景重复: 3 · 执行: 465 (NUWA 计算) 条件、限制与来源{
"condition": "standard prompting",
"temperature": "0.0 where exposed, otherwise provider default",
"tokenCapOverride": null,
"seed": null
}Constructed scenario distribution, not natural prevalence; no raw numerator inferred from rounded rate. 原文 ↗ · Table2 PDFp7; §3.6; Table9; text lines 00624,01738,01777,01789Original capture 2026-10-28T07:24:41.534922+00:00; sha256:b0b85491f4be9fb842c2f97891b033a75b44a85c756b0f3b92cf8da775d8f828 |
| Claude Sonnet 4.6 SVR | 0.219 比例 | 分母: 155 场景 · scenario mean after averaging 3 repeats 每场景重复: 3 · 执行: 465 (NUWA 计算) 条件、限制与来源{
"condition": "standard prompting",
"temperature": "0.0 where exposed, otherwise provider default",
"tokenCapOverride": null,
"seed": null
}Constructed scenario distribution, not natural prevalence; no raw numerator inferred from rounded rate. 原文 ↗ · Table2 PDFp7; §3.6; Table9; text lines 00624,01738,01777,01789Original capture 2026-10-28T07:24:41.534922+00:00; sha256:b0b85491f4be9fb842c2f97891b033a75b44a85c756b0f3b92cf8da775d8f828 |
| Claude Sonnet 4.6 ASR | 0.238 比例 | 分母: 105 攻击标注场景 · scenario mean after averaging 3 repeats 每场景重复: 3 · 执行: 315 (NUWA 计算) 条件、限制与来源{
"condition": "standard prompting",
"temperature": "0.0 where exposed, otherwise provider default",
"tokenCapOverride": null,
"seed": null
}Constructed scenario distribution, not natural prevalence; no raw numerator inferred from rounded rate. 原文 ↗ · Table2 PDFp7; §3.6; Table9; text lines 00624,01738,01777,01789Original capture 2026-10-28T07:24:41.534922+00:00; sha256:b0b85491f4be9fb842c2f97891b033a75b44a85c756b0f3b92cf8da775d8f828 |
| Claude Sonnet 4.6 STCR | 0.639 比例 | 分母: 155 场景 · scenario mean after averaging 3 repeats 每场景重复: 3 · 执行: 465 (NUWA 计算) 条件、限制与来源{
"condition": "standard prompting",
"temperature": "0.0 where exposed, otherwise provider default",
"tokenCapOverride": null,
"seed": null
}Constructed scenario distribution, not natural prevalence; no raw numerator inferred from rounded rate. 原文 ↗ · Table2 PDFp7; §3.6; Table9; text lines 00624,01738,01777,01789Original capture 2026-10-28T07:24:41.534922+00:00; sha256:b0b85491f4be9fb842c2f97891b033a75b44a85c756b0f3b92cf8da775d8f828 |
| Claude Sonnet 4.6 Latent | 0.194 比例 | 分母: 155 场景 · scenario mean after averaging 3 repeats 每场景重复: 3 · 执行: 465 (NUWA 计算) 条件、限制与来源{
"condition": "standard prompting",
"temperature": "0.0 where exposed, otherwise provider default",
"tokenCapOverride": null,
"seed": null
}Constructed scenario distribution, not natural prevalence; no raw numerator inferred from rounded rate. 原文 ↗ · Table2 PDFp7; §3.6; Table9; text lines 00624,01738,01777,01789Original capture 2026-10-28T07:24:41.534922+00:00; sha256:b0b85491f4be9fb842c2f97891b033a75b44a85c756b0f3b92cf8da775d8f828 |
| DeepSeek V4 Flash TSR | 0.852 比例 | 分母: 155 场景 · scenario mean after averaging 3 repeats 每场景重复: 3 · 执行: 465 (NUWA 计算) 条件、限制与来源{
"condition": "standard prompting",
"temperature": "0.0 where exposed, otherwise provider default",
"tokenCapOverride": null,
"seed": null
}Constructed scenario distribution, not natural prevalence; no raw numerator inferred from rounded rate. 原文 ↗ · Table2 PDFp7; §3.6; Table9; text lines 00625,01739,01773,01785Original capture 2026-10-28T07:24:41.534922+00:00; sha256:b0b85491f4be9fb842c2f97891b033a75b44a85c756b0f3b92cf8da775d8f828 |
| DeepSeek V4 Flash SVR | 0.4 比例 | 分母: 155 场景 · scenario mean after averaging 3 repeats 每场景重复: 3 · 执行: 465 (NUWA 计算) 条件、限制与来源{
"condition": "standard prompting",
"temperature": "0.0 where exposed, otherwise provider default",
"tokenCapOverride": null,
"seed": null
}Constructed scenario distribution, not natural prevalence; no raw numerator inferred from rounded rate. 原文 ↗ · Table2 PDFp7; §3.6; Table9; text lines 00625,01739,01773,01785Original capture 2026-10-28T07:24:41.534922+00:00; sha256:b0b85491f4be9fb842c2f97891b033a75b44a85c756b0f3b92cf8da775d8f828 |
| DeepSeek V4 Flash ASR | 0.533 比例 | 分母: 105 攻击标注场景 · scenario mean after averaging 3 repeats 每场景重复: 3 · 执行: 315 (NUWA 计算) 条件、限制与来源{
"condition": "standard prompting",
"temperature": "0.0 where exposed, otherwise provider default",
"tokenCapOverride": null,
"seed": null
}Constructed scenario distribution, not natural prevalence; no raw numerator inferred from rounded rate. 原文 ↗ · Table2 PDFp7; §3.6; Table9; text lines 00625,01739,01773,01785Original capture 2026-10-28T07:24:41.534922+00:00; sha256:b0b85491f4be9fb842c2f97891b033a75b44a85c756b0f3b92cf8da775d8f828 |
| DeepSeek V4 Flash STCR | 0.465 比例 | 分母: 155 场景 · scenario mean after averaging 3 repeats 每场景重复: 3 · 执行: 465 (NUWA 计算) 条件、限制与来源{
"condition": "standard prompting",
"temperature": "0.0 where exposed, otherwise provider default",
"tokenCapOverride": null,
"seed": null
}Constructed scenario distribution, not natural prevalence; no raw numerator inferred from rounded rate. 原文 ↗ · Table2 PDFp7; §3.6; Table9; text lines 00625,01739,01773,01785Original capture 2026-10-28T07:24:41.534922+00:00; sha256:b0b85491f4be9fb842c2f97891b033a75b44a85c756b0f3b92cf8da775d8f828 |
| DeepSeek V4 Flash Latent | 0.387 比例 | 分母: 155 场景 · scenario mean after averaging 3 repeats 每场景重复: 3 · 执行: 465 (NUWA 计算) 条件、限制与来源{
"condition": "standard prompting",
"temperature": "0.0 where exposed, otherwise provider default",
"tokenCapOverride": null,
"seed": null
}Constructed scenario distribution, not natural prevalence; no raw numerator inferred from rounded rate. 原文 ↗ · Table2 PDFp7; §3.6; Table9; text lines 00625,01739,01773,01785Original capture 2026-10-28T07:24:41.534922+00:00; sha256:b0b85491f4be9fb842c2f97891b033a75b44a85c756b0f3b92cf8da775d8f828 |
| DeepSeek V4 Pro TSR | 0.778 比例 | 分母: 155 场景 · scenario mean after averaging 3 repeats 每场景重复: 3 · 执行: 465 (NUWA 计算) 条件、限制与来源{
"condition": "standard prompting",
"temperature": "0.0 where exposed, otherwise provider default",
"tokenCapOverride": null,
"seed": null
}Constructed scenario distribution, not natural prevalence; no raw numerator inferred from rounded rate. 原文 ↗ · Table2 PDFp7; §3.6; Table9; text lines 00626,00862,01554,01740,01774,01786Original capture 2026-10-28T07:24:41.534922+00:00; sha256:b0b85491f4be9fb842c2f97891b033a75b44a85c756b0f3b92cf8da775d8f828 |
| DeepSeek V4 Pro SVR | 0.353 比例 | 分母: 155 场景 · scenario mean after averaging 3 repeats 每场景重复: 3 · 执行: 465 (NUWA 计算) 条件、限制与来源{
"condition": "standard prompting",
"temperature": "0.0 where exposed, otherwise provider default",
"tokenCapOverride": null,
"seed": null
}Constructed scenario distribution, not natural prevalence; no raw numerator inferred from rounded rate. 原文 ↗ · Table2 PDFp7; §3.6; Table9; text lines 00626,00862,01554,01740,01774,01786Original capture 2026-10-28T07:24:41.534922+00:00; sha256:b0b85491f4be9fb842c2f97891b033a75b44a85c756b0f3b92cf8da775d8f828 |
| DeepSeek V4 Pro ASR | 0.467 比例 | 分母: 105 攻击标注场景 · scenario mean after averaging 3 repeats 每场景重复: 3 · 执行: 315 (NUWA 计算) 条件、限制与来源{
"condition": "standard prompting",
"temperature": "0.0 where exposed, otherwise provider default",
"tokenCapOverride": null,
"seed": null
}Constructed scenario distribution, not natural prevalence; no raw numerator inferred from rounded rate. 原文 ↗ · Table2 PDFp7; §3.6; Table9; text lines 00626,00862,01554,01740,01774,01786Original capture 2026-10-28T07:24:41.534922+00:00; sha256:b0b85491f4be9fb842c2f97891b033a75b44a85c756b0f3b92cf8da775d8f828 |
| DeepSeek V4 Pro STCR | 0.516 比例 | 分母: 155 场景 · scenario mean after averaging 3 repeats 每场景重复: 3 · 执行: 465 (NUWA 计算) 条件、限制与来源{
"condition": "standard prompting",
"temperature": "0.0 where exposed, otherwise provider default",
"tokenCapOverride": null,
"seed": null
}Constructed scenario distribution, not natural prevalence; no raw numerator inferred from rounded rate. 原文 ↗ · Table2 PDFp7; §3.6; Table9; text lines 00626,00862,01554,01740,01774,01786Original capture 2026-10-28T07:24:41.534922+00:00; sha256:b0b85491f4be9fb842c2f97891b033a75b44a85c756b0f3b92cf8da775d8f828 |
| DeepSeek V4 Pro Latent | 0.327 比例 | 分母: 155 场景 · scenario mean after averaging 3 repeats 每场景重复: 3 · 执行: 465 (NUWA 计算) 条件、限制与来源{
"condition": "standard prompting",
"temperature": "0.0 where exposed, otherwise provider default",
"tokenCapOverride": null,
"seed": null
}Constructed scenario distribution, not natural prevalence; no raw numerator inferred from rounded rate. 原文 ↗ · Table2 PDFp7; §3.6; Table9; text lines 00626,00862,01554,01740,01774,01786Original capture 2026-10-28T07:24:41.534922+00:00; sha256:b0b85491f4be9fb842c2f97891b033a75b44a85c756b0f3b92cf8da775d8f828 |
| Gemini 3.1 Flash-Lite Preview TSR | 0.626 比例 | 分母: 155 场景 · scenario mean after averaging 3 repeats 每场景重复: 3 · 执行: 465 (NUWA 计算) 条件、限制与来源{
"condition": "standard prompting",
"temperature": "0.0 where exposed, otherwise provider default",
"tokenCapOverride": null,
"seed": null
}Constructed scenario distribution, not natural prevalence; no raw numerator inferred from rounded rate. 原文 ↗ · Table2 PDFp7; §3.6; Table9; text lines 00606,00627,01741Original capture 2026-10-28T07:24:41.534922+00:00; sha256:b0b85491f4be9fb842c2f97891b033a75b44a85c756b0f3b92cf8da775d8f828 |
| Gemini 3.1 Flash-Lite Preview SVR | 0.465 比例 | 分母: 155 场景 · scenario mean after averaging 3 repeats 每场景重复: 3 · 执行: 465 (NUWA 计算) 条件、限制与来源{
"condition": "standard prompting",
"temperature": "0.0 where exposed, otherwise provider default",
"tokenCapOverride": null,
"seed": null
}Constructed scenario distribution, not natural prevalence; no raw numerator inferred from rounded rate. 原文 ↗ · Table2 PDFp7; §3.6; Table9; text lines 00606,00627,01741Original capture 2026-10-28T07:24:41.534922+00:00; sha256:b0b85491f4be9fb842c2f97891b033a75b44a85c756b0f3b92cf8da775d8f828 |
| Gemini 3.1 Flash-Lite Preview ASR | 0.667 比例 | 分母: 105 攻击标注场景 · scenario mean after averaging 3 repeats 每场景重复: 3 · 执行: 315 (NUWA 计算) 条件、限制与来源{
"condition": "standard prompting",
"temperature": "0.0 where exposed, otherwise provider default",
"tokenCapOverride": null,
"seed": null
}Constructed scenario distribution, not natural prevalence; no raw numerator inferred from rounded rate. 原文 ↗ · Table2 PDFp7; §3.6; Table9; text lines 00606,00627,01741Original capture 2026-10-28T07:24:41.534922+00:00; sha256:b0b85491f4be9fb842c2f97891b033a75b44a85c756b0f3b92cf8da775d8f828 |
| Gemini 3.1 Flash-Lite Preview STCR | 0.31 比例 | 分母: 155 场景 · scenario mean after averaging 3 repeats 每场景重复: 3 · 执行: 465 (NUWA 计算) 条件、限制与来源{
"condition": "standard prompting",
"temperature": "0.0 where exposed, otherwise provider default",
"tokenCapOverride": null,
"seed": null
}Constructed scenario distribution, not natural prevalence; no raw numerator inferred from rounded rate. 原文 ↗ · Table2 PDFp7; §3.6; Table9; text lines 00606,00627,01741Original capture 2026-10-28T07:24:41.534922+00:00; sha256:b0b85491f4be9fb842c2f97891b033a75b44a85c756b0f3b92cf8da775d8f828 |
| Gemini 3.1 Flash-Lite Preview Latent | 0.432 比例 | 分母: 155 场景 · scenario mean after averaging 3 repeats 每场景重复: 3 · 执行: 465 (NUWA 计算) 条件、限制与来源{
"condition": "standard prompting",
"temperature": "0.0 where exposed, otherwise provider default",
"tokenCapOverride": null,
"seed": null
}Constructed scenario distribution, not natural prevalence; no raw numerator inferred from rounded rate. 原文 ↗ · Table2 PDFp7; §3.6; Table9; text lines 00606,00627,01741Original capture 2026-10-28T07:24:41.534922+00:00; sha256:b0b85491f4be9fb842c2f97891b033a75b44a85c756b0f3b92cf8da775d8f828 |
| Gemini 3.1 Pro Preview TSR | 0.82 比例 | 分母: 155 场景 · scenario mean after averaging 3 repeats 每场景重复: 3 · 执行: 465 (NUWA 计算) 条件、限制与来源{
"condition": "standard prompting",
"temperature": "0.0 where exposed, otherwise provider default",
"tokenCapOverride": null,
"seed": null
}Constructed scenario distribution, not natural prevalence; no raw numerator inferred from rounded rate. 原文 ↗ · Table2 PDFp7; §3.6; Table9; text lines 00628Original capture 2026-10-28T07:24:41.534922+00:00; sha256:b0b85491f4be9fb842c2f97891b033a75b44a85c756b0f3b92cf8da775d8f828 |
| Gemini 3.1 Pro Preview SVR | 0.266 比例 | 分母: 155 场景 · scenario mean after averaging 3 repeats 每场景重复: 3 · 执行: 465 (NUWA 计算) 条件、限制与来源{
"condition": "standard prompting",
"temperature": "0.0 where exposed, otherwise provider default",
"tokenCapOverride": null,
"seed": null
}Constructed scenario distribution, not natural prevalence; no raw numerator inferred from rounded rate. 原文 ↗ · Table2 PDFp7; §3.6; Table9; text lines 00628Original capture 2026-10-28T07:24:41.534922+00:00; sha256:b0b85491f4be9fb842c2f97891b033a75b44a85c756b0f3b92cf8da775d8f828 |
| Gemini 3.1 Pro Preview ASR | 0.402 比例 | 分母: 105 攻击标注场景 · scenario mean after averaging 3 repeats 每场景重复: 3 · 执行: 315 (NUWA 计算) 条件、限制与来源{
"condition": "standard prompting",
"temperature": "0.0 where exposed, otherwise provider default",
"tokenCapOverride": null,
"seed": null
}Constructed scenario distribution, not natural prevalence; no raw numerator inferred from rounded rate. 原文 ↗ · Table2 PDFp7; §3.6; Table9; text lines 00628Original capture 2026-10-28T07:24:41.534922+00:00; sha256:b0b85491f4be9fb842c2f97891b033a75b44a85c756b0f3b92cf8da775d8f828 |
| Gemini 3.1 Pro Preview STCR | 0.576 比例 | 分母: 155 场景 · scenario mean after averaging 3 repeats 每场景重复: 3 · 执行: 465 (NUWA 计算) 条件、限制与来源{
"condition": "standard prompting",
"temperature": "0.0 where exposed, otherwise provider default",
"tokenCapOverride": null,
"seed": null
}Constructed scenario distribution, not natural prevalence; no raw numerator inferred from rounded rate. 原文 ↗ · Table2 PDFp7; §3.6; Table9; text lines 00628Original capture 2026-10-28T07:24:41.534922+00:00; sha256:b0b85491f4be9fb842c2f97891b033a75b44a85c756b0f3b92cf8da775d8f828 |
| Gemini 3.1 Pro Preview Latent | 0.259 比例 | 分母: 155 场景 · scenario mean after averaging 3 repeats 每场景重复: 3 · 执行: 465 (NUWA 计算) 条件、限制与来源{
"condition": "standard prompting",
"temperature": "0.0 where exposed, otherwise provider default",
"tokenCapOverride": null,
"seed": null
}Constructed scenario distribution, not natural prevalence; no raw numerator inferred from rounded rate. 原文 ↗ · Table2 PDFp7; §3.6; Table9; text lines 00628Original capture 2026-10-28T07:24:41.534922+00:00; sha256:b0b85491f4be9fb842c2f97891b033a75b44a85c756b0f3b92cf8da775d8f828 |
| Gemini 3 Flash Preview TSR | 0.766 比例 | 分母: 155 场景 · scenario mean after averaging 3 repeats 每场景重复: 3 · 执行: 465 (NUWA 计算) 条件、限制与来源{
"condition": "standard prompting",
"temperature": "0.0 where exposed, otherwise provider default",
"tokenCapOverride": null,
"seed": null
}Constructed scenario distribution, not natural prevalence; no raw numerator inferred from rounded rate. 原文 ↗ · Table2 PDFp7; §3.6; Table9; text lines 00629,01742,01778,01790Original capture 2026-10-28T07:24:41.534922+00:00; sha256:b0b85491f4be9fb842c2f97891b033a75b44a85c756b0f3b92cf8da775d8f828 |
| Gemini 3 Flash Preview SVR | 0.383 比例 | 分母: 155 场景 · scenario mean after averaging 3 repeats 每场景重复: 3 · 执行: 465 (NUWA 计算) 条件、限制与来源{
"condition": "standard prompting",
"temperature": "0.0 where exposed, otherwise provider default",
"tokenCapOverride": null,
"seed": null
}Constructed scenario distribution, not natural prevalence; no raw numerator inferred from rounded rate. 原文 ↗ · Table2 PDFp7; §3.6; Table9; text lines 00629,01742,01778,01790Original capture 2026-10-28T07:24:41.534922+00:00; sha256:b0b85491f4be9fb842c2f97891b033a75b44a85c756b0f3b92cf8da775d8f828 |
| Gemini 3 Flash Preview ASR | 0.558 比例 | 分母: 105 攻击标注场景 · scenario mean after averaging 3 repeats 每场景重复: 3 · 执行: 315 (NUWA 计算) 条件、限制与来源{
"condition": "standard prompting",
"temperature": "0.0 where exposed, otherwise provider default",
"tokenCapOverride": null,
"seed": null
}Constructed scenario distribution, not natural prevalence; no raw numerator inferred from rounded rate. 原文 ↗ · Table2 PDFp7; §3.6; Table9; text lines 00629,01742,01778,01790Original capture 2026-10-28T07:24:41.534922+00:00; sha256:b0b85491f4be9fb842c2f97891b033a75b44a85c756b0f3b92cf8da775d8f828 |
| Gemini 3 Flash Preview STCR | 0.455 比例 | 分母: 155 场景 · scenario mean after averaging 3 repeats 每场景重复: 3 · 执行: 465 (NUWA 计算) 条件、限制与来源{
"condition": "standard prompting",
"temperature": "0.0 where exposed, otherwise provider default",
"tokenCapOverride": null,
"seed": null
}Constructed scenario distribution, not natural prevalence; no raw numerator inferred from rounded rate. 原文 ↗ · Table2 PDFp7; §3.6; Table9; text lines 00629,01742,01778,01790Original capture 2026-10-28T07:24:41.534922+00:00; sha256:b0b85491f4be9fb842c2f97891b033a75b44a85c756b0f3b92cf8da775d8f828 |
| Gemini 3 Flash Preview Latent | 0.377 比例 | 分母: 155 场景 · scenario mean after averaging 3 repeats 每场景重复: 3 · 执行: 465 (NUWA 计算) 条件、限制与来源{
"condition": "standard prompting",
"temperature": "0.0 where exposed, otherwise provider default",
"tokenCapOverride": null,
"seed": null
}Constructed scenario distribution, not natural prevalence; no raw numerator inferred from rounded rate. 原文 ↗ · Table2 PDFp7; §3.6; Table9; text lines 00629,01742,01778,01790Original capture 2026-10-28T07:24:41.534922+00:00; sha256:b0b85491f4be9fb842c2f97891b033a75b44a85c756b0f3b92cf8da775d8f828 |
| Gemma 4 31B-IT TSR | 0.761 比例 | 分母: 155 场景 · scenario mean after averaging 3 repeats 每场景重复: 3 · 执行: 465 (NUWA 计算) 条件、限制与来源{
"condition": "standard prompting",
"temperature": "0.0 where exposed, otherwise provider default",
"tokenCapOverride": null,
"seed": null
}Constructed scenario distribution, not natural prevalence; no raw numerator inferred from rounded rate. 原文 ↗ · Table2 PDFp7; §3.6; Table9; text lines 00630Original capture 2026-10-28T07:24:41.534922+00:00; sha256:b0b85491f4be9fb842c2f97891b033a75b44a85c756b0f3b92cf8da775d8f828 |
| Gemma 4 31B-IT SVR | 0.271 比例 | 分母: 155 场景 · scenario mean after averaging 3 repeats 每场景重复: 3 · 执行: 465 (NUWA 计算) 条件、限制与来源{
"condition": "standard prompting",
"temperature": "0.0 where exposed, otherwise provider default",
"tokenCapOverride": null,
"seed": null
}Constructed scenario distribution, not natural prevalence; no raw numerator inferred from rounded rate. 原文 ↗ · Table2 PDFp7; §3.6; Table9; text lines 00630Original capture 2026-10-28T07:24:41.534922+00:00; sha256:b0b85491f4be9fb842c2f97891b033a75b44a85c756b0f3b92cf8da775d8f828 |
| Gemma 4 31B-IT ASR | 0.381 比例 | 分母: 105 攻击标注场景 · scenario mean after averaging 3 repeats 每场景重复: 3 · 执行: 315 (NUWA 计算) 条件、限制与来源{
"condition": "standard prompting",
"temperature": "0.0 where exposed, otherwise provider default",
"tokenCapOverride": null,
"seed": null
}Constructed scenario distribution, not natural prevalence; no raw numerator inferred from rounded rate. 原文 ↗ · Table2 PDFp7; §3.6; Table9; text lines 00630Original capture 2026-10-28T07:24:41.534922+00:00; sha256:b0b85491f4be9fb842c2f97891b033a75b44a85c756b0f3b92cf8da775d8f828 |
| Gemma 4 31B-IT STCR | 0.587 比例 | 分母: 155 场景 · scenario mean after averaging 3 repeats 每场景重复: 3 · 执行: 465 (NUWA 计算) 条件、限制与来源{
"condition": "standard prompting",
"temperature": "0.0 where exposed, otherwise provider default",
"tokenCapOverride": null,
"seed": null
}Constructed scenario distribution, not natural prevalence; no raw numerator inferred from rounded rate. 原文 ↗ · Table2 PDFp7; §3.6; Table9; text lines 00630Original capture 2026-10-28T07:24:41.534922+00:00; sha256:b0b85491f4be9fb842c2f97891b033a75b44a85c756b0f3b92cf8da775d8f828 |
| Gemma 4 31B-IT Latent | 0.258 比例 | 分母: 155 场景 · scenario mean after averaging 3 repeats 每场景重复: 3 · 执行: 465 (NUWA 计算) 条件、限制与来源{
"condition": "standard prompting",
"temperature": "0.0 where exposed, otherwise provider default",
"tokenCapOverride": null,
"seed": null
}Constructed scenario distribution, not natural prevalence; no raw numerator inferred from rounded rate. 原文 ↗ · Table2 PDFp7; §3.6; Table9; text lines 00630Original capture 2026-10-28T07:24:41.534922+00:00; sha256:b0b85491f4be9fb842c2f97891b033a75b44a85c756b0f3b92cf8da775d8f828 |
| GPT-5.2 TSR | 0.761 比例 | 分母: 155 场景 · scenario mean after averaging 3 repeats 每场景重复: 3 · 执行: 465 (NUWA 计算) 条件、限制与来源{
"condition": "standard prompting",
"temperature": "0.0 where exposed, otherwise provider default",
"tokenCapOverride": null,
"seed": null
}Constructed scenario distribution, not natural prevalence; no raw numerator inferred from rounded rate. 原文 ↗ · Table2 PDFp7; §3.6; Table9; text lines 00631,01743Original capture 2026-10-28T07:24:41.534922+00:00; sha256:b0b85491f4be9fb842c2f97891b033a75b44a85c756b0f3b92cf8da775d8f828 |
| GPT-5.2 SVR | 0.097 比例 | 分母: 155 场景 · scenario mean after averaging 3 repeats 每场景重复: 3 · 执行: 465 (NUWA 计算) 条件、限制与来源{
"condition": "standard prompting",
"temperature": "0.0 where exposed, otherwise provider default",
"tokenCapOverride": null,
"seed": null
}Constructed scenario distribution, not natural prevalence; no raw numerator inferred from rounded rate. 原文 ↗ · Table2 PDFp7; §3.6; Table9; text lines 00631,01743Original capture 2026-10-28T07:24:41.534922+00:00; sha256:b0b85491f4be9fb842c2f97891b033a75b44a85c756b0f3b92cf8da775d8f828 |
| GPT-5.2 ASR | 0.143 比例 | 分母: 105 攻击标注场景 · scenario mean after averaging 3 repeats 每场景重复: 3 · 执行: 315 (NUWA 计算) 条件、限制与来源{
"condition": "standard prompting",
"temperature": "0.0 where exposed, otherwise provider default",
"tokenCapOverride": null,
"seed": null
}Constructed scenario distribution, not natural prevalence; no raw numerator inferred from rounded rate. 原文 ↗ · Table2 PDFp7; §3.6; Table9; text lines 00631,01743Original capture 2026-10-28T07:24:41.534922+00:00; sha256:b0b85491f4be9fb842c2f97891b033a75b44a85c756b0f3b92cf8da775d8f828 |
| GPT-5.2 STCR | 0.677 比例 | 分母: 155 场景 · scenario mean after averaging 3 repeats 每场景重复: 3 · 执行: 465 (NUWA 计算) 条件、限制与来源{
"condition": "standard prompting",
"temperature": "0.0 where exposed, otherwise provider default",
"tokenCapOverride": null,
"seed": null
}Constructed scenario distribution, not natural prevalence; no raw numerator inferred from rounded rate. 原文 ↗ · Table2 PDFp7; §3.6; Table9; text lines 00631,01743Original capture 2026-10-28T07:24:41.534922+00:00; sha256:b0b85491f4be9fb842c2f97891b033a75b44a85c756b0f3b92cf8da775d8f828 |
| GPT-5.2 Latent | 0.097 比例 | 分母: 155 场景 · scenario mean after averaging 3 repeats 每场景重复: 3 · 执行: 465 (NUWA 计算) 条件、限制与来源{
"condition": "standard prompting",
"temperature": "0.0 where exposed, otherwise provider default",
"tokenCapOverride": null,
"seed": null
}Constructed scenario distribution, not natural prevalence; no raw numerator inferred from rounded rate. 原文 ↗ · Table2 PDFp7; §3.6; Table9; text lines 00631,01743Original capture 2026-10-28T07:24:41.534922+00:00; sha256:b0b85491f4be9fb842c2f97891b033a75b44a85c756b0f3b92cf8da775d8f828 |
| GPT-5.5 TSR | 0.832 比例 | 分母: 155 场景 · scenario mean after averaging 3 repeats 每场景重复: 3 · 执行: 465 (NUWA 计算) 条件、限制与来源{
"condition": "standard prompting",
"temperature": "0.0 where exposed, otherwise provider default",
"tokenCapOverride": null,
"seed": null
}Constructed scenario distribution, not natural prevalence; no raw numerator inferred from rounded rate. 原文 ↗ · Table2 PDFp7; §3.6; Table9; text lines 00605,00632,01744,01775,01787Original capture 2026-10-28T07:24:41.534922+00:00; sha256:b0b85491f4be9fb842c2f97891b033a75b44a85c756b0f3b92cf8da775d8f828 |
| GPT-5.5 SVR | 0.058 比例 | 分母: 155 场景 · scenario mean after averaging 3 repeats 每场景重复: 3 · 执行: 465 (NUWA 计算) 条件、限制与来源{
"condition": "standard prompting",
"temperature": "0.0 where exposed, otherwise provider default",
"tokenCapOverride": null,
"seed": null
}Constructed scenario distribution, not natural prevalence; no raw numerator inferred from rounded rate. 原文 ↗ · Table2 PDFp7; §3.6; Table9; text lines 00605,00632,01744,01775,01787Original capture 2026-10-28T07:24:41.534922+00:00; sha256:b0b85491f4be9fb842c2f97891b033a75b44a85c756b0f3b92cf8da775d8f828 |
| GPT-5.5 ASR | 0.076 比例 | 分母: 105 攻击标注场景 · scenario mean after averaging 3 repeats 每场景重复: 3 · 执行: 315 (NUWA 计算) 条件、限制与来源{
"condition": "standard prompting",
"temperature": "0.0 where exposed, otherwise provider default",
"tokenCapOverride": null,
"seed": null
}Constructed scenario distribution, not natural prevalence; no raw numerator inferred from rounded rate. 原文 ↗ · Table2 PDFp7; §3.6; Table9; text lines 00605,00632,01744,01775,01787Original capture 2026-10-28T07:24:41.534922+00:00; sha256:b0b85491f4be9fb842c2f97891b033a75b44a85c756b0f3b92cf8da775d8f828 |
| GPT-5.5 STCR | 0.716 比例 | 分母: 155 场景 · scenario mean after averaging 3 repeats 每场景重复: 3 · 执行: 465 (NUWA 计算) 条件、限制与来源{
"condition": "standard prompting",
"temperature": "0.0 where exposed, otherwise provider default",
"tokenCapOverride": null,
"seed": null
}Constructed scenario distribution, not natural prevalence; no raw numerator inferred from rounded rate. 原文 ↗ · Table2 PDFp7; §3.6; Table9; text lines 00605,00632,01744,01775,01787Original capture 2026-10-28T07:24:41.534922+00:00; sha256:b0b85491f4be9fb842c2f97891b033a75b44a85c756b0f3b92cf8da775d8f828 |
| GPT-5.5 Latent | 0.045 比例 | 分母: 155 场景 · scenario mean after averaging 3 repeats 每场景重复: 3 · 执行: 465 (NUWA 计算) 条件、限制与来源{
"condition": "standard prompting",
"temperature": "0.0 where exposed, otherwise provider default",
"tokenCapOverride": null,
"seed": null
}Constructed scenario distribution, not natural prevalence; no raw numerator inferred from rounded rate. 原文 ↗ · Table2 PDFp7; §3.6; Table9; text lines 00605,00632,01744,01775,01787Original capture 2026-10-28T07:24:41.534922+00:00; sha256:b0b85491f4be9fb842c2f97891b033a75b44a85c756b0f3b92cf8da775d8f828 |
| GPT-5 mini TSR | 0.684 比例 | 分母: 155 场景 · scenario mean after averaging 3 repeats 每场景重复: 3 · 执行: 465 (NUWA 计算) 条件、限制与来源{
"condition": "standard prompting",
"temperature": "0.0 where exposed, otherwise provider default",
"tokenCapOverride": null,
"seed": null
}Constructed scenario distribution, not natural prevalence; no raw numerator inferred from rounded rate. 原文 ↗ · Table2 PDFp7; §3.6; Table9; text lines 00633,01745,01776,01788Original capture 2026-10-28T07:24:41.534922+00:00; sha256:b0b85491f4be9fb842c2f97891b033a75b44a85c756b0f3b92cf8da775d8f828 |
| GPT-5 mini SVR | 0.29 比例 | 分母: 155 场景 · scenario mean after averaging 3 repeats 每场景重复: 3 · 执行: 465 (NUWA 计算) 条件、限制与来源{
"condition": "standard prompting",
"temperature": "0.0 where exposed, otherwise provider default",
"tokenCapOverride": null,
"seed": null
}Constructed scenario distribution, not natural prevalence; no raw numerator inferred from rounded rate. 原文 ↗ · Table2 PDFp7; §3.6; Table9; text lines 00633,01745,01776,01788Original capture 2026-10-28T07:24:41.534922+00:00; sha256:b0b85491f4be9fb842c2f97891b033a75b44a85c756b0f3b92cf8da775d8f828 |
| GPT-5 mini ASR | 0.381 比例 | 分母: 105 攻击标注场景 · scenario mean after averaging 3 repeats 每场景重复: 3 · 执行: 315 (NUWA 计算) 条件、限制与来源{
"condition": "standard prompting",
"temperature": "0.0 where exposed, otherwise provider default",
"tokenCapOverride": null,
"seed": null
}Constructed scenario distribution, not natural prevalence; no raw numerator inferred from rounded rate. 原文 ↗ · Table2 PDFp7; §3.6; Table9; text lines 00633,01745,01776,01788Original capture 2026-10-28T07:24:41.534922+00:00; sha256:b0b85491f4be9fb842c2f97891b033a75b44a85c756b0f3b92cf8da775d8f828 |
| GPT-5 mini STCR | 0.503 比例 | 分母: 155 场景 · scenario mean after averaging 3 repeats 每场景重复: 3 · 执行: 465 (NUWA 计算) 条件、限制与来源{
"condition": "standard prompting",
"temperature": "0.0 where exposed, otherwise provider default",
"tokenCapOverride": null,
"seed": null
}Constructed scenario distribution, not natural prevalence; no raw numerator inferred from rounded rate. 原文 ↗ · Table2 PDFp7; §3.6; Table9; text lines 00633,01745,01776,01788Original capture 2026-10-28T07:24:41.534922+00:00; sha256:b0b85491f4be9fb842c2f97891b033a75b44a85c756b0f3b92cf8da775d8f828 |
| GPT-5 mini Latent | 0.271 比例 | 分母: 155 场景 · scenario mean after averaging 3 repeats 每场景重复: 3 · 执行: 465 (NUWA 计算) 条件、限制与来源{
"condition": "standard prompting",
"temperature": "0.0 where exposed, otherwise provider default",
"tokenCapOverride": null,
"seed": null
}Constructed scenario distribution, not natural prevalence; no raw numerator inferred from rounded rate. 原文 ↗ · Table2 PDFp7; §3.6; Table9; text lines 00633,01745,01776,01788Original capture 2026-10-28T07:24:41.534922+00:00; sha256:b0b85491f4be9fb842c2f97891b033a75b44a85c756b0f3b92cf8da775d8f828 |
| MiniMax M2.5 TSR | 0.858 比例 | 分母: 155 场景 · scenario mean after averaging 3 repeats 每场景重复: 3 · 执行: 465 (NUWA 计算) 条件、限制与来源{
"condition": "standard prompting",
"temperature": "0.0 where exposed, otherwise provider default",
"tokenCapOverride": null,
"seed": null
}Constructed scenario distribution, not natural prevalence; no raw numerator inferred from rounded rate. 原文 ↗ · Table2 PDFp7; §3.6; Table9; text lines 00634Original capture 2026-10-28T07:24:41.534922+00:00; sha256:b0b85491f4be9fb842c2f97891b033a75b44a85c756b0f3b92cf8da775d8f828 |
| MiniMax M2.5 SVR | 0.161 比例 | 分母: 155 场景 · scenario mean after averaging 3 repeats 每场景重复: 3 · 执行: 465 (NUWA 计算) 条件、限制与来源{
"condition": "standard prompting",
"temperature": "0.0 where exposed, otherwise provider default",
"tokenCapOverride": null,
"seed": null
}Constructed scenario distribution, not natural prevalence; no raw numerator inferred from rounded rate. 原文 ↗ · Table2 PDFp7; §3.6; Table9; text lines 00634Original capture 2026-10-28T07:24:41.534922+00:00; sha256:b0b85491f4be9fb842c2f97891b033a75b44a85c756b0f3b92cf8da775d8f828 |
| MiniMax M2.5 ASR | 0.219 比例 | 分母: 105 攻击标注场景 · scenario mean after averaging 3 repeats 每场景重复: 3 · 执行: 315 (NUWA 计算) 条件、限制与来源{
"condition": "standard prompting",
"temperature": "0.0 where exposed, otherwise provider default",
"tokenCapOverride": null,
"seed": null
}Constructed scenario distribution, not natural prevalence; no raw numerator inferred from rounded rate. 原文 ↗ · Table2 PDFp7; §3.6; Table9; text lines 00634Original capture 2026-10-28T07:24:41.534922+00:00; sha256:b0b85491f4be9fb842c2f97891b033a75b44a85c756b0f3b92cf8da775d8f828 |
| MiniMax M2.5 STCR | 0.729 比例 | 分母: 155 场景 · scenario mean after averaging 3 repeats 每场景重复: 3 · 执行: 465 (NUWA 计算) 条件、限制与来源{
"condition": "standard prompting",
"temperature": "0.0 where exposed, otherwise provider default",
"tokenCapOverride": null,
"seed": null
}Constructed scenario distribution, not natural prevalence; no raw numerator inferred from rounded rate. 原文 ↗ · Table2 PDFp7; §3.6; Table9; text lines 00634Original capture 2026-10-28T07:24:41.534922+00:00; sha256:b0b85491f4be9fb842c2f97891b033a75b44a85c756b0f3b92cf8da775d8f828 |
| MiniMax M2.5 Latent | 0.142 比例 | 分母: 155 场景 · scenario mean after averaging 3 repeats 每场景重复: 3 · 执行: 465 (NUWA 计算) 条件、限制与来源{
"condition": "standard prompting",
"temperature": "0.0 where exposed, otherwise provider default",
"tokenCapOverride": null,
"seed": null
}Constructed scenario distribution, not natural prevalence; no raw numerator inferred from rounded rate. 原文 ↗ · Table2 PDFp7; §3.6; Table9; text lines 00634Original capture 2026-10-28T07:24:41.534922+00:00; sha256:b0b85491f4be9fb842c2f97891b033a75b44a85c756b0f3b92cf8da775d8f828 |
| Mean TSR | 0.784 比例0.742–0.825 · scenario-percentile-bootstrap · 水平未说明 | 条件、限制与来源model mean over 12 headline models; reminder deltas over9paired models Scenario-level composition uncertainty; confidence level not inferred from elsewhere in paper. 原文 ↗ · Table17 PDFp18; Appendix H; text lines 00602,01678Original capture 2026-10-28T07:24:41.534922+00:00; sha256:b0b85491f4be9fb842c2f97891b033a75b44a85c756b0f3b92cf8da775d8f828 |
| Mean SVR | 0.256 比例0.213–0.3 · scenario-percentile-bootstrap · 水平未说明 | 条件、限制与来源model mean over 12 headline models; reminder deltas over9paired models Scenario-level composition uncertainty; confidence level not inferred from elsewhere in paper. 原文 ↗ · Table17 PDFp18; Appendix H; text lines 01679Original capture 2026-10-28T07:24:41.534922+00:00; sha256:b0b85491f4be9fb842c2f97891b033a75b44a85c756b0f3b92cf8da775d8f828 |
| Mean ASR | 0.347 比例0.294–0.4 · scenario-percentile-bootstrap · 水平未说明 | 条件、限制与来源model mean over 12 headline models; reminder deltas over9paired models Scenario-level composition uncertainty; confidence level not inferred from elsewhere in paper. 原文 ↗ · Table17 PDFp18; Appendix H; text lines 00153,00603,01680Original capture 2026-10-28T07:24:41.534922+00:00; sha256:b0b85491f4be9fb842c2f97891b033a75b44a85c756b0f3b92cf8da775d8f828 |
| Mean STCR | 0.577 比例0.527–0.627 · scenario-percentile-bootstrap · 水平未说明 | 条件、限制与来源model mean over 12 headline models; reminder deltas over9paired models Scenario-level composition uncertainty; confidence level not inferred from elsewhere in paper. 原文 ↗ · Table17 PDFp18; Appendix H; text lines 01681Original capture 2026-10-28T07:24:41.534922+00:00; sha256:b0b85491f4be9fb842c2f97891b033a75b44a85c756b0f3b92cf8da775d8f828 |
| Mean latent violation rate | 0.24 比例0.199–0.284 · scenario-percentile-bootstrap · 水平未说明 | 条件、限制与来源model mean over 12 headline models; reminder deltas over9paired models Scenario-level composition uncertainty; confidence level not inferred from elsewhere in paper. 原文 ↗ · Table17 PDFp18; Appendix H; text lines 00859,01682Original capture 2026-10-28T07:24:41.534922+00:00; sha256:b0b85491f4be9fb842c2f97891b033a75b44a85c756b0f3b92cf8da775d8f828 |
| Internal-authority SVR | 0.424 比例0.347–0.499 · scenario-percentile-bootstrap · 水平未说明 | 条件、限制与来源model mean over 12 headline models; reminder deltas over9paired models Scenario-level composition uncertainty; confidence level not inferred from elsewhere in paper. 原文 ↗ · Table17 PDFp18; Appendix H; text lines 01683Original capture 2026-10-28T07:24:41.534922+00:00; sha256:b0b85491f4be9fb842c2f97891b033a75b44a85c756b0f3b92cf8da775d8f828 |
| External-attack SVR | 0.242 比例0.184–0.305 · scenario-percentile-bootstrap · 水平未说明 | 条件、限制与来源model mean over 12 headline models; reminder deltas over9paired models Scenario-level composition uncertainty; confidence level not inferred from elsewhere in paper. 原文 ↗ · Table17 PDFp18; Appendix H; text lines 01684Original capture 2026-10-28T07:24:41.534922+00:00; sha256:b0b85491f4be9fb842c2f97891b033a75b44a85c756b0f3b92cf8da775d8f828 |
| Internal minus external SVR | 0.182 比例差0.083–0.278 · scenario-percentile-bootstrap · 水平未说明 | 条件、限制与来源model mean over 12 headline models; reminder deltas over9paired models Scenario-level composition uncertainty; confidence level not inferred from elsewhere in paper. 原文 ↗ · Table17 PDFp18; Appendix H; text lines 01685Original capture 2026-10-28T07:24:41.534922+00:00; sha256:b0b85491f4be9fb842c2f97891b033a75b44a85c756b0f3b92cf8da775d8f828 |
| Prompt reminder∆ASR | -0.107 比例差-0.132–-0.083 · scenario-percentile-bootstrap · 水平未说明 | 条件、限制与来源model mean over 12 headline models; reminder deltas over9paired models Scenario-level composition uncertainty; confidence level not inferred from elsewhere in paper. 原文 ↗ · Table17 PDFp18; Appendix H; text lines 01689Original capture 2026-10-28T07:24:41.534922+00:00; sha256:b0b85491f4be9fb842c2f97891b033a75b44a85c756b0f3b92cf8da775d8f828 |
关联阅读
NUWA 阅读关联；不据此声称原文引用、实验复现或成绩等价。
任务与测量
Benchmark 协议档案
- 任务／对象
- 155 个可执行沙箱场景分为非对抗失效、内部权限受损和外部攻击。12 个模型各运行三次，共 5,580 次执行。指标先在场景内平均重复，再跨适用场景平均；ASR 只使用 105 个攻击标注场景，其余主指标使用 155 个场景。
- 测量目标
- 终态安全是否掩盖中途违规？
- 解释边界
- ASR 仅针对攻击标记子集；回放重执行既定工具调用，不重新运行模型，不能证明反事实因果。
测量档案 · NUWA 解读
档案区分来源方法阅读与既有目录信息；已经公开不代表完成独立证据复核或双语编辑复核。
- 测量构念
- 终态安全是否掩盖中途违规？
- 任务与样本
- 155 个可执行沙箱场景分为非对抗失效、内部权限受损和外部攻击。12 个模型各运行三次，共 5,580 次执行。指标先在场景内平均重复，再跨适用场景平均；ASR 只使用 105 个攻击标注场景，其余主指标使用 155 个场景。
- 判分与失效情形
- TSR 是任务通过率，SVR 是终点安全违规率，ASR 是攻击标注场景上的安全违规率；STCR 要求任务成功且无终点或中间违规。潜在违规指被禁止的中间轨迹事件。ASR 分母为 105 个攻击标注场景，其余主指标为 155；每场景先平均三次重复。
- 系统条件
- 来源特定接口与构造任务条件；此次阅读不确立与其他协议的匹配可比性。
- 统计含义
- 逐指标保留证据表的总体、重复和区间定义；未给出的事件分子保持未知。
- 效度与解释
- ASR 仅针对攻击标记子集；回放重执行既定工具调用，不重新运行模型，不能证明反事实因果。
- 适用研究问题
- NUWA 建议：新增过程证据层；保留主试验、配对提醒、压力测试和回放各自分母。
- TraceProbe: An Execution-Based Benchmark for Diagnosing Safety Failures in LLM Agents · §3.6; §5–6; Tables 2,15,17; limitations; text lines 00483Original resource captured 2026-10-28T07:24:41.534922+00:00; sha256:b0b85491f4be9fb842c2f97891b033a75b44a85c756b0f3b92cf8da775d8f828
- TraceProbe: An Execution-Based Benchmark for Diagnosing Safety Failures in LLM Agents · §3.2 illustrative case-dispute scenario; Table 24Original resource captured 2026-10-28T07:24:41.534922+00:00; sha256:b0b85491f4be9fb842c2f97891b033a75b44a85c756b0f3b92cf8da775d8f828
原始来源图表与材料
原始材料选读已形成测量档案。已转录数值保留在私有记录证据表；本批未新增获批图表或导入论文图。
相关研究问题
网络安全 / 非可信输入导致智能体受侵害 →
编辑范围关联；不同测量的成绩不等价。