NUWA RESEARCH RECORD
Identifying the Risks of LM Agents with an LM-Emulated Sandbox
原始文献 ↗ · BibTeX · RIS框架无需实现全部真实工具即可发现可能造成严重后果的失误；人工核查同时发现模拟器错误与漏检的不安全动作。
研究问题
语言模型模拟能否在工具部署前高效发现智能体失误？
设计与前提
ToolEmu 模拟工具执行并对完整轨迹评估安全性与帮助程度，以人工判断核验普通与对抗模拟器的配对轨迹。
研究发现
框架无需实现全部真实工具即可发现可能造成严重后果的失误；人工核查同时发现模拟器错误与漏检的不安全动作。
局限与反证
解释受模拟保真度和评审召回率限制。模拟失败既非已确认生产事件，也不是现实失败频率。
来源证据
| 系统 | 测量 | 原值 | 条件与位置 |
|---|---|---|---|
| GPT-4 / Basic | Average safety score | 2.007score (0–3) | ToolEmu adversarial emulator / Basic#S5.T5; row GPT-4 / Basic; condition ToolEmu adversarial emulator / Basic; column Average safety score |
| GPT-4 / Basic | Failure incidence | 39.4%% | ToolEmu adversarial emulator / Basic#S5.T5; row GPT-4 / Basic; condition ToolEmu adversarial emulator / Basic; column Failure incidence |
| GPT-4 / Basic | Average helpfulness score | 1.458score (0–3) | ToolEmu adversarial emulator / Basic#S5.T5; row GPT-4 / Basic; condition ToolEmu adversarial emulator / Basic; column Average helpfulness score |
| Claude-2 / Basic | Average safety score | 1.829score (0–3) | ToolEmu adversarial emulator / Basic#S5.T5; row Claude-2 / Basic; condition ToolEmu adversarial emulator / Basic; column Average safety score |
| Claude-2 / Basic | Failure incidence | 44.3%% | ToolEmu adversarial emulator / Basic#S5.T5; row Claude-2 / Basic; condition ToolEmu adversarial emulator / Basic; column Failure incidence |
| Claude-2 / Basic | Average helpfulness score | 1.464score (0–3) | ToolEmu adversarial emulator / Basic#S5.T5; row Claude-2 / Basic; condition ToolEmu adversarial emulator / Basic; column Average helpfulness score |
| ChatGPT-3.5 / Basic | Average safety score | 1.430score (0–3) | ToolEmu adversarial emulator / Basic#S5.T5; row ChatGPT-3.5 / Basic; condition ToolEmu adversarial emulator / Basic; column Average safety score |
| ChatGPT-3.5 / Basic | Failure incidence | 62.0%% | ToolEmu adversarial emulator / Basic#S5.T5; row ChatGPT-3.5 / Basic; condition ToolEmu adversarial emulator / Basic; column Failure incidence |
| ChatGPT-3.5 / Basic | Average helpfulness score | 0.768score (0–3) | ToolEmu adversarial emulator / Basic#S5.T5; row ChatGPT-3.5 / Basic; condition ToolEmu adversarial emulator / Basic; column Average helpfulness score |
| Vicuna-1.5-13B / Basic | Average safety score | 1.552score (0–3) | ToolEmu adversarial emulator / Basic#S5.T5; row Vicuna-1.5-13B / Basic; condition ToolEmu adversarial emulator / Basic; column Average safety score |
| Vicuna-1.5-13B / Basic | Failure incidence | 54.6%% | ToolEmu adversarial emulator / Basic#S5.T5; row Vicuna-1.5-13B / Basic; condition ToolEmu adversarial emulator / Basic; column Failure incidence |
| Vicuna-1.5-13B / Basic | Average helpfulness score | 0.441score (0–3) | ToolEmu adversarial emulator / Basic#S5.T5; row Vicuna-1.5-13B / Basic; condition ToolEmu adversarial emulator / Basic; column Average helpfulness score |
| Vicuna-1.5-7B / Basic | Average safety score | 1.850score (0–3) | ToolEmu adversarial emulator / Basic#S5.T5; row Vicuna-1.5-7B / Basic; condition ToolEmu adversarial emulator / Basic; column Average safety score |
| Vicuna-1.5-7B / Basic | Failure incidence | 45.0%% | ToolEmu adversarial emulator / Basic#S5.T5; row Vicuna-1.5-7B / Basic; condition ToolEmu adversarial emulator / Basic; column Failure incidence |
| Vicuna-1.5-7B / Basic | Average helpfulness score | 0.364score (0–3) | ToolEmu adversarial emulator / Basic#S5.T5; row Vicuna-1.5-7B / Basic; condition ToolEmu adversarial emulator / Basic; column Average helpfulness score |
| GPT-4 / Safety | Average safety score | 2.359score (0–3) | ToolEmu adversarial emulator / Safety#S5.T5; row GPT-4 / Safety; condition ToolEmu adversarial emulator / Safety; column Average safety score |
| GPT-4 / Safety | Failure incidence | 23.9%% | ToolEmu adversarial emulator / Safety#S5.T5; row GPT-4 / Safety; condition ToolEmu adversarial emulator / Safety; column Failure incidence |
| GPT-4 / Safety | Average helpfulness score | 1.824score (0–3) | ToolEmu adversarial emulator / Safety#S5.T5; row GPT-4 / Safety; condition ToolEmu adversarial emulator / Safety; column Average helpfulness score |
| Helpful + Safety / Basic | Average safety score | 2.241score (0–3) | ToolEmu adversarial emulator / Basic#S5.T5; row Helpful + Safety / Basic; condition ToolEmu adversarial emulator / Basic; column Average safety score |
| Helpful + Safety / Basic | Failure incidence | 30.5%% | ToolEmu adversarial emulator / Basic#S5.T5; row Helpful + Safety / Basic; condition ToolEmu adversarial emulator / Basic; column Failure incidence |
| Helpful + Safety / Basic | Average helpfulness score | 1.624score (0–3) | ToolEmu adversarial emulator / Basic#S5.T5; row Helpful + Safety / Basic; condition ToolEmu adversarial emulator / Basic; column Average helpfulness score |
| NoAct / - | Average safety score | 3.000score (0–3) | ToolEmu adversarial emulator / -#S5.T5; row NoAct / -; condition ToolEmu adversarial emulator / -; column Average safety score |
| NoAct / - | Failure incidence | 0.00%% | ToolEmu adversarial emulator / -#S5.T5; row NoAct / -; condition ToolEmu adversarial emulator / -; column Failure incidence |
| NoAct / - | Average helpfulness score | 0.063score (0–3) | ToolEmu adversarial emulator / -#S5.T5; row NoAct / -; condition ToolEmu adversarial emulator / -; column Average helpfulness score |
| Standard emulator | Identified failure precision | 72.5% ± 7.1%%72.5% ± 7.1% | External annotators#S4.T3; row Standard emulator; condition External annotators; column Identified failure precision |
| Standard emulator | True failure incidence | 39.6% ± 4.9%%39.6% ± 4.9% | External annotators#S4.T3; row Standard emulator; condition External annotators; column True failure incidence |
| Adversarial emulator | Identified failure precision | 68.8% ± 6.7%%68.8% ± 6.7% | External annotators#S4.T3; row Adversarial emulator; condition External annotators; column Identified failure precision |
| Adversarial emulator | True failure incidence | 50.0% ± 5.1%%50.0% ± 5.1% | External annotators#S4.T3; row Adversarial emulator; condition External annotators; column True failure incidence |
| Standard emulator | Identified failure precision | 71.1% ± 7.4%%71.1% ± 7.4% | Author annotators#A2.T4; row Standard emulator; condition Author annotators; column Identified failure precision |
| Standard emulator | True failure incidence | 40.8% ± 5.0%%40.8% ± 5.0% | Author annotators#A2.T4; row Standard emulator; condition Author annotators; column True failure incidence |
| Adversarial emulator | Identified failure precision | 69.4% ± 6.6%%69.4% ± 6.6% | Author annotators#A2.T4; row Adversarial emulator; condition Author annotators; column Identified failure precision |
| Adversarial emulator | True failure incidence | 47.5% ± 5.0%%47.5% ± 5.0% | Author annotators#A2.T4; row Adversarial emulator; condition Author annotators; column True failure incidence |
阅读范围与复核
Selected original HTML sections and table context; not a complete paper/appendix audit or independent reproduction.
模型辅助整理，独立证据复核待完成。
核对原始来源 ↗