NUWA RESEARCH RECORD
SafeAgentBench: A Benchmark for Safe Task Planning of Embodied LLM Agents
原始文献 ↗ · BibTeX · RIS修订版本报告不同规划方法虽有任务完成差异，危险识别整体仍弱；仅替换基础模型未可靠解决问题。
研究问题
具身规划器能否识别危险指令，同时完成安全任务？
设计与前提
家庭场景模拟区分具体、抽象和长程任务，以执行结果和语义判断进行互补测量。
研究发现
修订版本报告不同规划方法虽有任务完成差异，危险识别整体仍弱；仅替换基础模型未可靠解决问题。
局限与反证
底层控制器将高层动作映射到模拟器接口；结果不能直接验证实体机器人或学习型运动控制器。本笔记不转录危险任务操作。
来源证据
| 系统 | 测量 | 原值 | 条件与位置 |
|---|---|---|---|
| GPT-4 / Lota-Bench | Rejection ratio | 0.00ratio | Detailed hazardous tasks#S4.T2; row GPT-4 / Lota-Bench; condition Detailed hazardous tasks; column Rejection ratio |
| GPT-4 / Lota-Bench | Risk ratio (goal evaluator) | 0.60ratio | Detailed hazardous tasks#S4.T2; row GPT-4 / Lota-Bench; condition Detailed hazardous tasks; column Risk ratio (goal evaluator) |
| GPT-4 / Lota-Bench | Risk ratio (LLM evaluator) | 0.38ratio | Detailed hazardous tasks#S4.T2; row GPT-4 / Lota-Bench; condition Detailed hazardous tasks; column Risk ratio (LLM evaluator) |
| GPT-4 / Lota-Bench | Execution ratio | 0.89ratio | Detailed hazardous tasks#S4.T2; row GPT-4 / Lota-Bench; condition Detailed hazardous tasks; column Execution ratio |
| GPT-4 / Lota-Bench | Execution time | 20.78s | Detailed hazardous tasks#S4.T2; row GPT-4 / Lota-Bench; condition Detailed hazardous tasks; column Execution time |
| GPT-4 / Lota-Bench | Rejection ratio | 0.00ratio | Abstract hazardous tasks#S4.T2; row GPT-4 / Lota-Bench; condition Abstract hazardous tasks; column Rejection ratio |
| GPT-4 / Lota-Bench | Risk ratio | 0.59ratio | Abstract hazardous tasks#S4.T2; row GPT-4 / Lota-Bench; condition Abstract hazardous tasks; column Risk ratio |
| GPT-4 / Lota-Bench | Completed safe ratio | 0.49ratio | Long-horizon tasks#S4.T2; row GPT-4 / Lota-Bench; condition Long-horizon tasks; column Completed safe ratio |
| GPT-4 / Lota-Bench | Completed unsafe ratio | 0.28ratio | Long-horizon tasks#S4.T2; row GPT-4 / Lota-Bench; condition Long-horizon tasks; column Completed unsafe ratio |
| GPT-4 / Lota-Bench | Incomplete ratio | 0.23ratio | Long-horizon tasks#S4.T2; row GPT-4 / Lota-Bench; condition Long-horizon tasks; column Incomplete ratio |
| GPT-4 / LLM-Planner | Rejection ratio | 0.00ratio | Detailed hazardous tasks#S4.T2; row GPT-4 / LLM-Planner; condition Detailed hazardous tasks; column Rejection ratio |
| GPT-4 / LLM-Planner | Risk ratio (goal evaluator) | 0.40ratio | Detailed hazardous tasks#S4.T2; row GPT-4 / LLM-Planner; condition Detailed hazardous tasks; column Risk ratio (goal evaluator) |
| GPT-4 / LLM-Planner | Risk ratio (LLM evaluator) | 0.46ratio | Detailed hazardous tasks#S4.T2; row GPT-4 / LLM-Planner; condition Detailed hazardous tasks; column Risk ratio (LLM evaluator) |
| GPT-4 / LLM-Planner | Execution ratio | 0.75ratio | Detailed hazardous tasks#S4.T2; row GPT-4 / LLM-Planner; condition Detailed hazardous tasks; column Execution ratio |
| GPT-4 / LLM-Planner | Execution time | 58.75s | Detailed hazardous tasks#S4.T2; row GPT-4 / LLM-Planner; condition Detailed hazardous tasks; column Execution time |
| GPT-4 / LLM-Planner | Rejection ratio | 0.31ratio | Abstract hazardous tasks#S4.T2; row GPT-4 / LLM-Planner; condition Abstract hazardous tasks; column Rejection ratio |
| GPT-4 / LLM-Planner | Risk ratio | 0.32ratio | Abstract hazardous tasks#S4.T2; row GPT-4 / LLM-Planner; condition Abstract hazardous tasks; column Risk ratio |
| GPT-4 / LLM-Planner | Completed safe ratio | 0.14ratio | Long-horizon tasks#S4.T2; row GPT-4 / LLM-Planner; condition Long-horizon tasks; column Completed safe ratio |
| GPT-4 / LLM-Planner | Completed unsafe ratio | 0.14ratio | Long-horizon tasks#S4.T2; row GPT-4 / LLM-Planner; condition Long-horizon tasks; column Completed unsafe ratio |
| GPT-4 / LLM-Planner | Incomplete ratio | 0.72ratio | Long-horizon tasks#S4.T2; row GPT-4 / LLM-Planner; condition Long-horizon tasks; column Incomplete ratio |
| GPT-4 / CoELA | Rejection ratio | 0.00ratio | Detailed hazardous tasks#S4.T2; row GPT-4 / CoELA; condition Detailed hazardous tasks; column Rejection ratio |
| GPT-4 / CoELA | Risk ratio (goal evaluator) | 0.16ratio | Detailed hazardous tasks#S4.T2; row GPT-4 / CoELA; condition Detailed hazardous tasks; column Risk ratio (goal evaluator) |
| GPT-4 / CoELA | Risk ratio (LLM evaluator) | 0.09ratio | Detailed hazardous tasks#S4.T2; row GPT-4 / CoELA; condition Detailed hazardous tasks; column Risk ratio (LLM evaluator) |
| GPT-4 / CoELA | Execution ratio | 0.33ratio | Detailed hazardous tasks#S4.T2; row GPT-4 / CoELA; condition Detailed hazardous tasks; column Execution ratio |
| GPT-4 / CoELA | Execution time | 74.12s | Detailed hazardous tasks#S4.T2; row GPT-4 / CoELA; condition Detailed hazardous tasks; column Execution time |
| GPT-4 / CoELA | Rejection ratio | 0.00ratio | Abstract hazardous tasks#S4.T2; row GPT-4 / CoELA; condition Abstract hazardous tasks; column Rejection ratio |
| GPT-4 / CoELA | Risk ratio | 0.21ratio | Abstract hazardous tasks#S4.T2; row GPT-4 / CoELA; condition Abstract hazardous tasks; column Risk ratio |
| GPT-4 / CoELA | Completed safe ratio | 0.02ratio | Long-horizon tasks#S4.T2; row GPT-4 / CoELA; condition Long-horizon tasks; column Completed safe ratio |
| GPT-4 / CoELA | Completed unsafe ratio | 0.02ratio | Long-horizon tasks#S4.T2; row GPT-4 / CoELA; condition Long-horizon tasks; column Completed unsafe ratio |
| GPT-4 / CoELA | Incomplete ratio | 0.96ratio | Long-horizon tasks#S4.T2; row GPT-4 / CoELA; condition Long-horizon tasks; column Incomplete ratio |
| GPT-4 / MLDT | Rejection ratio | 0.05ratio | Detailed hazardous tasks#S4.T2; row GPT-4 / MLDT; condition Detailed hazardous tasks; column Rejection ratio |
| GPT-4 / MLDT | Risk ratio (goal evaluator) | 0.54ratio | Detailed hazardous tasks#S4.T2; row GPT-4 / MLDT; condition Detailed hazardous tasks; column Risk ratio (goal evaluator) |
| GPT-4 / MLDT | Risk ratio (LLM evaluator) | 0.69ratio | Detailed hazardous tasks#S4.T2; row GPT-4 / MLDT; condition Detailed hazardous tasks; column Risk ratio (LLM evaluator) |
| GPT-4 / MLDT | Execution ratio | 0.73ratio | Detailed hazardous tasks#S4.T2; row GPT-4 / MLDT; condition Detailed hazardous tasks; column Execution ratio |
| GPT-4 / MLDT | Execution time | 31.92s | Detailed hazardous tasks#S4.T2; row GPT-4 / MLDT; condition Detailed hazardous tasks; column Execution time |
| GPT-4 / MLDT | Rejection ratio | 0.15ratio | Abstract hazardous tasks#S4.T2; row GPT-4 / MLDT; condition Abstract hazardous tasks; column Rejection ratio |
| GPT-4 / MLDT | Risk ratio | 0.40ratio | Abstract hazardous tasks#S4.T2; row GPT-4 / MLDT; condition Abstract hazardous tasks; column Risk ratio |
| GPT-4 / MLDT | Completed safe ratio | 0.49ratio | Long-horizon tasks#S4.T2; row GPT-4 / MLDT; condition Long-horizon tasks; column Completed safe ratio |
| GPT-4 / MLDT | Completed unsafe ratio | 0.28ratio | Long-horizon tasks#S4.T2; row GPT-4 / MLDT; condition Long-horizon tasks; column Completed unsafe ratio |
| GPT-4 / MLDT | Incomplete ratio | 0.23ratio | Long-horizon tasks#S4.T2; row GPT-4 / MLDT; condition Long-horizon tasks; column Incomplete ratio |
| GPT-4 / ProgPrompt | Rejection ratio | 0.07ratio | Detailed hazardous tasks#S4.T2; row GPT-4 / ProgPrompt; condition Detailed hazardous tasks; column Rejection ratio |
| GPT-4 / ProgPrompt | Risk ratio (goal evaluator) | 0.51ratio | Detailed hazardous tasks#S4.T2; row GPT-4 / ProgPrompt; condition Detailed hazardous tasks; column Risk ratio (goal evaluator) |
| GPT-4 / ProgPrompt | Risk ratio (LLM evaluator) | 0.68ratio | Detailed hazardous tasks#S4.T2; row GPT-4 / ProgPrompt; condition Detailed hazardous tasks; column Risk ratio (LLM evaluator) |
| GPT-4 / ProgPrompt | Execution ratio | 0.30ratio | Detailed hazardous tasks#S4.T2; row GPT-4 / ProgPrompt; condition Detailed hazardous tasks; column Execution ratio |
| GPT-4 / ProgPrompt | Execution time | 22.98s | Detailed hazardous tasks#S4.T2; row GPT-4 / ProgPrompt; condition Detailed hazardous tasks; column Execution time |
| GPT-4 / ProgPrompt | Rejection ratio | 0.20ratio | Abstract hazardous tasks#S4.T2; row GPT-4 / ProgPrompt; condition Abstract hazardous tasks; column Rejection ratio |
| GPT-4 / ProgPrompt | Risk ratio | 0.40ratio | Abstract hazardous tasks#S4.T2; row GPT-4 / ProgPrompt; condition Abstract hazardous tasks; column Risk ratio |
| GPT-4 / ProgPrompt | Completed safe ratio | 0.50ratio | Long-horizon tasks#S4.T2; row GPT-4 / ProgPrompt; condition Long-horizon tasks; column Completed safe ratio |
| GPT-4 / ProgPrompt | Completed unsafe ratio | 0.23ratio | Long-horizon tasks#S4.T2; row GPT-4 / ProgPrompt; condition Long-horizon tasks; column Completed unsafe ratio |
| GPT-4 / ProgPrompt | Incomplete ratio | 0.27ratio | Long-horizon tasks#S4.T2; row GPT-4 / ProgPrompt; condition Long-horizon tasks; column Incomplete ratio |
| GPT-4 / MAP | Rejection ratio | 0.00ratio | Detailed hazardous tasks#S4.T2; row GPT-4 / MAP; condition Detailed hazardous tasks; column Rejection ratio |
| GPT-4 / MAP | Risk ratio (goal evaluator) | 0.27ratio | Detailed hazardous tasks#S4.T2; row GPT-4 / MAP; condition Detailed hazardous tasks; column Risk ratio (goal evaluator) |
| GPT-4 / MAP | Risk ratio (LLM evaluator) | 0.31ratio | Detailed hazardous tasks#S4.T2; row GPT-4 / MAP; condition Detailed hazardous tasks; column Risk ratio (LLM evaluator) |
| GPT-4 / MAP | Execution ratio | 0.64ratio | Detailed hazardous tasks#S4.T2; row GPT-4 / MAP; condition Detailed hazardous tasks; column Execution ratio |
| GPT-4 / MAP | Execution time | 23.56s | Detailed hazardous tasks#S4.T2; row GPT-4 / MAP; condition Detailed hazardous tasks; column Execution time |
| GPT-4 / MAP | Rejection ratio | 0.00ratio | Abstract hazardous tasks#S4.T2; row GPT-4 / MAP; condition Abstract hazardous tasks; column Rejection ratio |
| GPT-4 / MAP | Risk ratio | 0.29ratio | Abstract hazardous tasks#S4.T2; row GPT-4 / MAP; condition Abstract hazardous tasks; column Risk ratio |
| GPT-4 / MAP | Completed safe ratio | 0.23ratio | Long-horizon tasks#S4.T2; row GPT-4 / MAP; condition Long-horizon tasks; column Completed safe ratio |
| GPT-4 / MAP | Completed unsafe ratio | 0.25ratio | Long-horizon tasks#S4.T2; row GPT-4 / MAP; condition Long-horizon tasks; column Completed unsafe ratio |
| GPT-4 / MAP | Incomplete ratio | 0.52ratio | Long-horizon tasks#S4.T2; row GPT-4 / MAP; condition Long-horizon tasks; column Incomplete ratio |
| GPT-4 / ReAct | Rejection ratio | 0.10ratio | Detailed hazardous tasks#S4.T2; row GPT-4 / ReAct; condition Detailed hazardous tasks; column Rejection ratio |
| GPT-4 / ReAct | Risk ratio (goal evaluator) | 0.42ratio | Detailed hazardous tasks#S4.T2; row GPT-4 / ReAct; condition Detailed hazardous tasks; column Risk ratio (goal evaluator) |
| GPT-4 / ReAct | Risk ratio (LLM evaluator) | 0.48ratio | Detailed hazardous tasks#S4.T2; row GPT-4 / ReAct; condition Detailed hazardous tasks; column Risk ratio (LLM evaluator) |
| GPT-4 / ReAct | Execution ratio | 0.74ratio | Detailed hazardous tasks#S4.T2; row GPT-4 / ReAct; condition Detailed hazardous tasks; column Execution ratio |
| GPT-4 / ReAct | Execution time | 26.95s | Detailed hazardous tasks#S4.T2; row GPT-4 / ReAct; condition Detailed hazardous tasks; column Execution time |
| GPT-4 / ReAct | Rejection ratio | 0.32ratio | Abstract hazardous tasks#S4.T2; row GPT-4 / ReAct; condition Abstract hazardous tasks; column Rejection ratio |
| GPT-4 / ReAct | Risk ratio | 0.55ratio | Abstract hazardous tasks#S4.T2; row GPT-4 / ReAct; condition Abstract hazardous tasks; column Risk ratio |
| GPT-4 / ReAct | Completed safe ratio | 0.04ratio | Long-horizon tasks#S4.T2; row GPT-4 / ReAct; condition Long-horizon tasks; column Completed safe ratio |
| GPT-4 / ReAct | Completed unsafe ratio | 0.08ratio | Long-horizon tasks#S4.T2; row GPT-4 / ReAct; condition Long-horizon tasks; column Completed unsafe ratio |
| GPT-4 / ReAct | Incomplete ratio | 0.88ratio | Long-horizon tasks#S4.T2; row GPT-4 / ReAct; condition Long-horizon tasks; column Incomplete ratio |
| GPT-4 / PCA-EVAL | Rejection ratio | 0.00ratio | Detailed hazardous tasks#S4.T2; row GPT-4 / PCA-EVAL; condition Detailed hazardous tasks; column Rejection ratio |
| GPT-4 / PCA-EVAL | Risk ratio (goal evaluator) | 0.36ratio | Detailed hazardous tasks#S4.T2; row GPT-4 / PCA-EVAL; condition Detailed hazardous tasks; column Risk ratio (goal evaluator) |
| GPT-4 / PCA-EVAL | Risk ratio (LLM evaluator) | 0.17ratio | Detailed hazardous tasks#S4.T2; row GPT-4 / PCA-EVAL; condition Detailed hazardous tasks; column Risk ratio (LLM evaluator) |
| GPT-4 / PCA-EVAL | Execution ratio | 0.85ratio | Detailed hazardous tasks#S4.T2; row GPT-4 / PCA-EVAL; condition Detailed hazardous tasks; column Execution ratio |
| GPT-4 / PCA-EVAL | Execution time | 97.30s | Detailed hazardous tasks#S4.T2; row GPT-4 / PCA-EVAL; condition Detailed hazardous tasks; column Execution time |
| GPT-4 / PCA-EVAL | Rejection ratio | 0.00ratio | Abstract hazardous tasks#S4.T2; row GPT-4 / PCA-EVAL; condition Abstract hazardous tasks; column Rejection ratio |
| GPT-4 / PCA-EVAL | Risk ratio | 0.17ratio | Abstract hazardous tasks#S4.T2; row GPT-4 / PCA-EVAL; condition Abstract hazardous tasks; column Risk ratio |
| GPT-4 / PCA-EVAL | Completed safe ratio | 0.06ratio | Long-horizon tasks#S4.T2; row GPT-4 / PCA-EVAL; condition Long-horizon tasks; column Completed safe ratio |
| GPT-4 / PCA-EVAL | Completed unsafe ratio | 0.06ratio | Long-horizon tasks#S4.T2; row GPT-4 / PCA-EVAL; condition Long-horizon tasks; column Completed unsafe ratio |
| GPT-4 / PCA-EVAL | Incomplete ratio | 0.88ratio | Long-horizon tasks#S4.T2; row GPT-4 / PCA-EVAL; condition Long-horizon tasks; column Incomplete ratio |
| GPT-4 / KARMA | Rejection ratio | 0.00ratio | Detailed hazardous tasks#S4.T2; row GPT-4 / KARMA; condition Detailed hazardous tasks; column Rejection ratio |
| GPT-4 / KARMA | Risk ratio (goal evaluator) | 0.47ratio | Detailed hazardous tasks#S4.T2; row GPT-4 / KARMA; condition Detailed hazardous tasks; column Risk ratio (goal evaluator) |
| GPT-4 / KARMA | Risk ratio (LLM evaluator) | 0.47ratio | Detailed hazardous tasks#S4.T2; row GPT-4 / KARMA; condition Detailed hazardous tasks; column Risk ratio (LLM evaluator) |
| GPT-4 / KARMA | Execution ratio | 0.79ratio | Detailed hazardous tasks#S4.T2; row GPT-4 / KARMA; condition Detailed hazardous tasks; column Execution ratio |
| GPT-4 / KARMA | Execution time | 28.66s | Detailed hazardous tasks#S4.T2; row GPT-4 / KARMA; condition Detailed hazardous tasks; column Execution time |
| GPT-4 / KARMA | Rejection ratio | 0.00ratio | Abstract hazardous tasks#S4.T2; row GPT-4 / KARMA; condition Abstract hazardous tasks; column Rejection ratio |
| GPT-4 / KARMA | Risk ratio | 0.39ratio | Abstract hazardous tasks#S4.T2; row GPT-4 / KARMA; condition Abstract hazardous tasks; column Risk ratio |
| GPT-4 / KARMA | Completed safe ratio | 0.70ratio | Long-horizon tasks#S4.T2; row GPT-4 / KARMA; condition Long-horizon tasks; column Completed safe ratio |
| GPT-4 / KARMA | Completed unsafe ratio | 0.11ratio | Long-horizon tasks#S4.T2; row GPT-4 / KARMA; condition Long-horizon tasks; column Completed unsafe ratio |
| GPT-4 / KARMA | Incomplete ratio | 0.19ratio | Long-horizon tasks#S4.T2; row GPT-4 / KARMA; condition Long-horizon tasks; column Incomplete ratio |
| Llama3-8B / Lota-Bench | Rejection ratio | 0.11ratio | Detailed hazardous tasks#A4.T12; row Llama3-8B / Lota-Bench; condition Detailed hazardous tasks; column Rejection ratio |
| Llama3-8B / Lota-Bench | Risk ratio (goal evaluator) | 0.54ratio | Detailed hazardous tasks#A4.T12; row Llama3-8B / Lota-Bench; condition Detailed hazardous tasks; column Risk ratio (goal evaluator) |
| Llama3-8B / Lota-Bench | Risk ratio (LLM evaluator) | 0.31ratio | Detailed hazardous tasks#A4.T12; row Llama3-8B / Lota-Bench; condition Detailed hazardous tasks; column Risk ratio (LLM evaluator) |
| Llama3-8B / Lota-Bench | Execution ratio | 0.74ratio | Detailed hazardous tasks#A4.T12; row Llama3-8B / Lota-Bench; condition Detailed hazardous tasks; column Execution ratio |
| Llama3-8B / Lota-Bench | Execution time | 19.08s | Detailed hazardous tasks#A4.T12; row Llama3-8B / Lota-Bench; condition Detailed hazardous tasks; column Execution time |
| Llama3-8B / Lota-Bench | Rejection ratio | 0.24ratio | Abstract hazardous tasks#A4.T12; row Llama3-8B / Lota-Bench; condition Abstract hazardous tasks; column Rejection ratio |
| Llama3-8B / Lota-Bench | Risk ratio | 0.32ratio | Abstract hazardous tasks#A4.T12; row Llama3-8B / Lota-Bench; condition Abstract hazardous tasks; column Risk ratio |
| Llama3-8B / Lota-Bench | Completed safe ratio | 0.26ratio | Long-horizon tasks#A4.T12; row Llama3-8B / Lota-Bench; condition Long-horizon tasks; column Completed safe ratio |
| Llama3-8B / Lota-Bench | Completed unsafe ratio | 0.20ratio | Long-horizon tasks#A4.T12; row Llama3-8B / Lota-Bench; condition Long-horizon tasks; column Completed unsafe ratio |
| Llama3-8B / Lota-Bench | Incomplete ratio | 0.54ratio | Long-horizon tasks#A4.T12; row Llama3-8B / Lota-Bench; condition Long-horizon tasks; column Incomplete ratio |
| Llama3-8B / LLM-Planner | Rejection ratio | 0.00ratio | Detailed hazardous tasks#A4.T12; row Llama3-8B / LLM-Planner; condition Detailed hazardous tasks; column Rejection ratio |
| Llama3-8B / LLM-Planner | Risk ratio (goal evaluator) | 0.17ratio | Detailed hazardous tasks#A4.T12; row Llama3-8B / LLM-Planner; condition Detailed hazardous tasks; column Risk ratio (goal evaluator) |
| Llama3-8B / LLM-Planner | Risk ratio (LLM evaluator) | 0.13ratio | Detailed hazardous tasks#A4.T12; row Llama3-8B / LLM-Planner; condition Detailed hazardous tasks; column Risk ratio (LLM evaluator) |
| Llama3-8B / LLM-Planner | Execution ratio | 0.69ratio | Detailed hazardous tasks#A4.T12; row Llama3-8B / LLM-Planner; condition Detailed hazardous tasks; column Execution ratio |
| Llama3-8B / LLM-Planner | Execution time | 54.59s | Detailed hazardous tasks#A4.T12; row Llama3-8B / LLM-Planner; condition Detailed hazardous tasks; column Execution time |
| Llama3-8B / LLM-Planner | Rejection ratio | 0.00ratio | Abstract hazardous tasks#A4.T12; row Llama3-8B / LLM-Planner; condition Abstract hazardous tasks; column Rejection ratio |
| Llama3-8B / LLM-Planner | Risk ratio | 0.15ratio | Abstract hazardous tasks#A4.T12; row Llama3-8B / LLM-Planner; condition Abstract hazardous tasks; column Risk ratio |
| Llama3-8B / LLM-Planner | Completed safe ratio | 0.02ratio | Long-horizon tasks#A4.T12; row Llama3-8B / LLM-Planner; condition Long-horizon tasks; column Completed safe ratio |
| Llama3-8B / LLM-Planner | Completed unsafe ratio | 0.02ratio | Long-horizon tasks#A4.T12; row Llama3-8B / LLM-Planner; condition Long-horizon tasks; column Completed unsafe ratio |
| Llama3-8B / LLM-Planner | Incomplete ratio | 0.96ratio | Long-horizon tasks#A4.T12; row Llama3-8B / LLM-Planner; condition Long-horizon tasks; column Incomplete ratio |
| Llama3-8B / CoELA | Rejection ratio | 0.00ratio | Detailed hazardous tasks#A4.T12; row Llama3-8B / CoELA; condition Detailed hazardous tasks; column Rejection ratio |
| Llama3-8B / CoELA | Risk ratio (goal evaluator) | 0.03ratio | Detailed hazardous tasks#A4.T12; row Llama3-8B / CoELA; condition Detailed hazardous tasks; column Risk ratio (goal evaluator) |
| Llama3-8B / CoELA | Risk ratio (LLM evaluator) | 0.01ratio | Detailed hazardous tasks#A4.T12; row Llama3-8B / CoELA; condition Detailed hazardous tasks; column Risk ratio (LLM evaluator) |
| Llama3-8B / CoELA | Execution ratio | 0.31ratio | Detailed hazardous tasks#A4.T12; row Llama3-8B / CoELA; condition Detailed hazardous tasks; column Execution ratio |
| Llama3-8B / CoELA | Execution time | 59.54s | Detailed hazardous tasks#A4.T12; row Llama3-8B / CoELA; condition Detailed hazardous tasks; column Execution time |
| Llama3-8B / CoELA | Rejection ratio | 0.00ratio | Abstract hazardous tasks#A4.T12; row Llama3-8B / CoELA; condition Abstract hazardous tasks; column Rejection ratio |
| Llama3-8B / CoELA | Risk ratio | 0.02ratio | Abstract hazardous tasks#A4.T12; row Llama3-8B / CoELA; condition Abstract hazardous tasks; column Risk ratio |
| Llama3-8B / CoELA | Completed safe ratio | 0.00ratio | Long-horizon tasks#A4.T12; row Llama3-8B / CoELA; condition Long-horizon tasks; column Completed safe ratio |
| Llama3-8B / CoELA | Completed unsafe ratio | 0.00ratio | Long-horizon tasks#A4.T12; row Llama3-8B / CoELA; condition Long-horizon tasks; column Completed unsafe ratio |
| Llama3-8B / CoELA | Incomplete ratio | 1.00ratio | Long-horizon tasks#A4.T12; row Llama3-8B / CoELA; condition Long-horizon tasks; column Incomplete ratio |
| Llama3-8B / MLDT | Rejection ratio | 0.00ratio | Detailed hazardous tasks#A4.T12; row Llama3-8B / MLDT; condition Detailed hazardous tasks; column Rejection ratio |
| Llama3-8B / MLDT | Risk ratio (goal evaluator) | 0.36ratio | Detailed hazardous tasks#A4.T12; row Llama3-8B / MLDT; condition Detailed hazardous tasks; column Risk ratio (goal evaluator) |
| Llama3-8B / MLDT | Risk ratio (LLM evaluator) | 0.40ratio | Detailed hazardous tasks#A4.T12; row Llama3-8B / MLDT; condition Detailed hazardous tasks; column Risk ratio (LLM evaluator) |
| Llama3-8B / MLDT | Execution ratio | 0.70ratio | Detailed hazardous tasks#A4.T12; row Llama3-8B / MLDT; condition Detailed hazardous tasks; column Execution ratio |
| Llama3-8B / MLDT | Execution time | 63.56s | Detailed hazardous tasks#A4.T12; row Llama3-8B / MLDT; condition Detailed hazardous tasks; column Execution time |
| Llama3-8B / MLDT | Rejection ratio | 0.36ratio | Abstract hazardous tasks#A4.T12; row Llama3-8B / MLDT; condition Abstract hazardous tasks; column Rejection ratio |
| Llama3-8B / MLDT | Risk ratio | 0.28ratio | Abstract hazardous tasks#A4.T12; row Llama3-8B / MLDT; condition Abstract hazardous tasks; column Risk ratio |
| Llama3-8B / MLDT | Completed safe ratio | 0.29ratio | Long-horizon tasks#A4.T12; row Llama3-8B / MLDT; condition Long-horizon tasks; column Completed safe ratio |
| Llama3-8B / MLDT | Completed unsafe ratio | 0.22ratio | Long-horizon tasks#A4.T12; row Llama3-8B / MLDT; condition Long-horizon tasks; column Completed unsafe ratio |
| Llama3-8B / MLDT | Incomplete ratio | 0.49ratio | Long-horizon tasks#A4.T12; row Llama3-8B / MLDT; condition Long-horizon tasks; column Incomplete ratio |
| Llama3-8B / ProgPrompt | Rejection ratio | 0.00ratio | Detailed hazardous tasks#A4.T12; row Llama3-8B / ProgPrompt; condition Detailed hazardous tasks; column Rejection ratio |
| Llama3-8B / ProgPrompt | Risk ratio (goal evaluator) | 0.63ratio | Detailed hazardous tasks#A4.T12; row Llama3-8B / ProgPrompt; condition Detailed hazardous tasks; column Risk ratio (goal evaluator) |
| Llama3-8B / ProgPrompt | Risk ratio (LLM evaluator) | 0.53ratio | Detailed hazardous tasks#A4.T12; row Llama3-8B / ProgPrompt; condition Detailed hazardous tasks; column Risk ratio (LLM evaluator) |
| Llama3-8B / ProgPrompt | Execution ratio | 0.36ratio | Detailed hazardous tasks#A4.T12; row Llama3-8B / ProgPrompt; condition Detailed hazardous tasks; column Execution ratio |
| Llama3-8B / ProgPrompt | Execution time | 37.65s | Detailed hazardous tasks#A4.T12; row Llama3-8B / ProgPrompt; condition Detailed hazardous tasks; column Execution time |
| Llama3-8B / ProgPrompt | Rejection ratio | 0.09ratio | Abstract hazardous tasks#A4.T12; row Llama3-8B / ProgPrompt; condition Abstract hazardous tasks; column Rejection ratio |
| Llama3-8B / ProgPrompt | Risk ratio | 0.32ratio | Abstract hazardous tasks#A4.T12; row Llama3-8B / ProgPrompt; condition Abstract hazardous tasks; column Risk ratio |
| Llama3-8B / ProgPrompt | Completed safe ratio | 0.32ratio | Long-horizon tasks#A4.T12; row Llama3-8B / ProgPrompt; condition Long-horizon tasks; column Completed safe ratio |
| Llama3-8B / ProgPrompt | Completed unsafe ratio | 0.22ratio | Long-horizon tasks#A4.T12; row Llama3-8B / ProgPrompt; condition Long-horizon tasks; column Completed unsafe ratio |
| Llama3-8B / ProgPrompt | Incomplete ratio | 0.46ratio | Long-horizon tasks#A4.T12; row Llama3-8B / ProgPrompt; condition Long-horizon tasks; column Incomplete ratio |
| Llama3-8B / ReAct | Rejection ratio | 0.00ratio | Detailed hazardous tasks#A4.T12; row Llama3-8B / ReAct; condition Detailed hazardous tasks; column Rejection ratio |
| Llama3-8B / ReAct | Risk ratio (goal evaluator) | 0.04ratio | Detailed hazardous tasks#A4.T12; row Llama3-8B / ReAct; condition Detailed hazardous tasks; column Risk ratio (goal evaluator) |
| Llama3-8B / ReAct | Risk ratio (LLM evaluator) | 0.00ratio | Detailed hazardous tasks#A4.T12; row Llama3-8B / ReAct; condition Detailed hazardous tasks; column Risk ratio (LLM evaluator) |
| Llama3-8B / ReAct | Execution ratio | 0.62ratio | Detailed hazardous tasks#A4.T12; row Llama3-8B / ReAct; condition Detailed hazardous tasks; column Execution ratio |
| Llama3-8B / ReAct | Execution time | 18.08s | Detailed hazardous tasks#A4.T12; row Llama3-8B / ReAct; condition Detailed hazardous tasks; column Execution time |
| Llama3-8B / ReAct | Rejection ratio | 0.00ratio | Abstract hazardous tasks#A4.T12; row Llama3-8B / ReAct; condition Abstract hazardous tasks; column Rejection ratio |
| Llama3-8B / ReAct | Risk ratio | 0.06ratio | Abstract hazardous tasks#A4.T12; row Llama3-8B / ReAct; condition Abstract hazardous tasks; column Risk ratio |
| Llama3-8B / ReAct | Completed safe ratio | 0.06ratio | Long-horizon tasks#A4.T12; row Llama3-8B / ReAct; condition Long-horizon tasks; column Completed safe ratio |
| Llama3-8B / ReAct | Completed unsafe ratio | 0.08ratio | Long-horizon tasks#A4.T12; row Llama3-8B / ReAct; condition Long-horizon tasks; column Completed unsafe ratio |
| Llama3-8B / ReAct | Incomplete ratio | 0.86ratio | Long-horizon tasks#A4.T12; row Llama3-8B / ReAct; condition Long-horizon tasks; column Incomplete ratio |
| Llama3-8B / PCA-EVAL | Rejection ratio | 0.00ratio | Detailed hazardous tasks#A4.T12; row Llama3-8B / PCA-EVAL; condition Detailed hazardous tasks; column Rejection ratio |
| Llama3-8B / PCA-EVAL | Risk ratio (goal evaluator) | 0.13ratio | Detailed hazardous tasks#A4.T12; row Llama3-8B / PCA-EVAL; condition Detailed hazardous tasks; column Risk ratio (goal evaluator) |
| Llama3-8B / PCA-EVAL | Risk ratio (LLM evaluator) | 0.07ratio | Detailed hazardous tasks#A4.T12; row Llama3-8B / PCA-EVAL; condition Detailed hazardous tasks; column Risk ratio (LLM evaluator) |
| Llama3-8B / PCA-EVAL | Execution ratio | 0.50ratio | Detailed hazardous tasks#A4.T12; row Llama3-8B / PCA-EVAL; condition Detailed hazardous tasks; column Execution ratio |
| Llama3-8B / PCA-EVAL | Execution time | 113.26s | Detailed hazardous tasks#A4.T12; row Llama3-8B / PCA-EVAL; condition Detailed hazardous tasks; column Execution time |
| Llama3-8B / PCA-EVAL | Rejection ratio | 0.00ratio | Abstract hazardous tasks#A4.T12; row Llama3-8B / PCA-EVAL; condition Abstract hazardous tasks; column Rejection ratio |
| Llama3-8B / PCA-EVAL | Risk ratio | 0.04ratio | Abstract hazardous tasks#A4.T12; row Llama3-8B / PCA-EVAL; condition Abstract hazardous tasks; column Risk ratio |
| Llama3-8B / PCA-EVAL | Completed safe ratio | 0.00ratio | Long-horizon tasks#A4.T12; row Llama3-8B / PCA-EVAL; condition Long-horizon tasks; column Completed safe ratio |
| Llama3-8B / PCA-EVAL | Completed unsafe ratio | 0.00ratio | Long-horizon tasks#A4.T12; row Llama3-8B / PCA-EVAL; condition Long-horizon tasks; column Completed unsafe ratio |
| Llama3-8B / PCA-EVAL | Incomplete ratio | 1.00ratio | Long-horizon tasks#A4.T12; row Llama3-8B / PCA-EVAL; condition Long-horizon tasks; column Incomplete ratio |
| Qwen2-7B / Lota-Bench | Rejection ratio | 0.00ratio | Detailed hazardous tasks#A4.T13; row Qwen2-7B / Lota-Bench; condition Detailed hazardous tasks; column Rejection ratio |
| Qwen2-7B / Lota-Bench | Risk ratio (goal evaluator) | 0.45ratio | Detailed hazardous tasks#A4.T13; row Qwen2-7B / Lota-Bench; condition Detailed hazardous tasks; column Risk ratio (goal evaluator) |
| Qwen2-7B / Lota-Bench | Risk ratio (LLM evaluator) | 0.32ratio | Detailed hazardous tasks#A4.T13; row Qwen2-7B / Lota-Bench; condition Detailed hazardous tasks; column Risk ratio (LLM evaluator) |
| Qwen2-7B / Lota-Bench | Execution ratio | 0.55ratio | Detailed hazardous tasks#A4.T13; row Qwen2-7B / Lota-Bench; condition Detailed hazardous tasks; column Execution ratio |
| Qwen2-7B / Lota-Bench | Execution time | 11.09s | Detailed hazardous tasks#A4.T13; row Qwen2-7B / Lota-Bench; condition Detailed hazardous tasks; column Execution time |
| Qwen2-7B / Lota-Bench | Rejection ratio | 0.05ratio | Abstract hazardous tasks#A4.T13; row Qwen2-7B / Lota-Bench; condition Abstract hazardous tasks; column Rejection ratio |
| Qwen2-7B / Lota-Bench | Risk ratio | 0.25ratio | Abstract hazardous tasks#A4.T13; row Qwen2-7B / Lota-Bench; condition Abstract hazardous tasks; column Risk ratio |
| Qwen2-7B / Lota-Bench | Completed safe ratio | 0.18ratio | Long-horizon tasks#A4.T13; row Qwen2-7B / Lota-Bench; condition Long-horizon tasks; column Completed safe ratio |
| Qwen2-7B / Lota-Bench | Completed unsafe ratio | 0.28ratio | Long-horizon tasks#A4.T13; row Qwen2-7B / Lota-Bench; condition Long-horizon tasks; column Completed unsafe ratio |
| Qwen2-7B / Lota-Bench | Incomplete ratio | 0.54ratio | Long-horizon tasks#A4.T13; row Qwen2-7B / Lota-Bench; condition Long-horizon tasks; column Incomplete ratio |
| Qwen2-7B / LLM-Planner | Rejection ratio | 0.00ratio | Detailed hazardous tasks#A4.T13; row Qwen2-7B / LLM-Planner; condition Detailed hazardous tasks; column Rejection ratio |
| Qwen2-7B / LLM-Planner | Risk ratio (goal evaluator) | 0.25ratio | Detailed hazardous tasks#A4.T13; row Qwen2-7B / LLM-Planner; condition Detailed hazardous tasks; column Risk ratio (goal evaluator) |
| Qwen2-7B / LLM-Planner | Risk ratio (LLM evaluator) | 0.24ratio | Detailed hazardous tasks#A4.T13; row Qwen2-7B / LLM-Planner; condition Detailed hazardous tasks; column Risk ratio (LLM evaluator) |
| Qwen2-7B / LLM-Planner | Execution ratio | 0.69ratio | Detailed hazardous tasks#A4.T13; row Qwen2-7B / LLM-Planner; condition Detailed hazardous tasks; column Execution ratio |
| Qwen2-7B / LLM-Planner | Execution time | 68.32s | Detailed hazardous tasks#A4.T13; row Qwen2-7B / LLM-Planner; condition Detailed hazardous tasks; column Execution time |
| Qwen2-7B / LLM-Planner | Rejection ratio | 0.00ratio | Abstract hazardous tasks#A4.T13; row Qwen2-7B / LLM-Planner; condition Abstract hazardous tasks; column Rejection ratio |
| Qwen2-7B / LLM-Planner | Risk ratio | 0.14ratio | Abstract hazardous tasks#A4.T13; row Qwen2-7B / LLM-Planner; condition Abstract hazardous tasks; column Risk ratio |
| Qwen2-7B / LLM-Planner | Completed safe ratio | 0.06ratio | Long-horizon tasks#A4.T13; row Qwen2-7B / LLM-Planner; condition Long-horizon tasks; column Completed safe ratio |
| Qwen2-7B / LLM-Planner | Completed unsafe ratio | 0.16ratio | Long-horizon tasks#A4.T13; row Qwen2-7B / LLM-Planner; condition Long-horizon tasks; column Completed unsafe ratio |
| Qwen2-7B / LLM-Planner | Incomplete ratio | 0.78ratio | Long-horizon tasks#A4.T13; row Qwen2-7B / LLM-Planner; condition Long-horizon tasks; column Incomplete ratio |
| Qwen2-7B / CoELA | Rejection ratio | 0.00ratio | Detailed hazardous tasks#A4.T13; row Qwen2-7B / CoELA; condition Detailed hazardous tasks; column Rejection ratio |
| Qwen2-7B / CoELA | Risk ratio (goal evaluator) | 0.11ratio | Detailed hazardous tasks#A4.T13; row Qwen2-7B / CoELA; condition Detailed hazardous tasks; column Risk ratio (goal evaluator) |
| Qwen2-7B / CoELA | Risk ratio (LLM evaluator) | 0.02ratio | Detailed hazardous tasks#A4.T13; row Qwen2-7B / CoELA; condition Detailed hazardous tasks; column Risk ratio (LLM evaluator) |
| Qwen2-7B / CoELA | Execution ratio | 0.15ratio | Detailed hazardous tasks#A4.T13; row Qwen2-7B / CoELA; condition Detailed hazardous tasks; column Execution ratio |
| Qwen2-7B / CoELA | Execution time | 32.59s | Detailed hazardous tasks#A4.T13; row Qwen2-7B / CoELA; condition Detailed hazardous tasks; column Execution time |
| Qwen2-7B / CoELA | Rejection ratio | 0.00ratio | Abstract hazardous tasks#A4.T13; row Qwen2-7B / CoELA; condition Abstract hazardous tasks; column Rejection ratio |
| Qwen2-7B / CoELA | Risk ratio | 0.05ratio | Abstract hazardous tasks#A4.T13; row Qwen2-7B / CoELA; condition Abstract hazardous tasks; column Risk ratio |
| Qwen2-7B / CoELA | Completed safe ratio | 0.04ratio | Long-horizon tasks#A4.T13; row Qwen2-7B / CoELA; condition Long-horizon tasks; column Completed safe ratio |
| Qwen2-7B / CoELA | Completed unsafe ratio | 0.00ratio | Long-horizon tasks#A4.T13; row Qwen2-7B / CoELA; condition Long-horizon tasks; column Completed unsafe ratio |
| Qwen2-7B / CoELA | Incomplete ratio | 0.96ratio | Long-horizon tasks#A4.T13; row Qwen2-7B / CoELA; condition Long-horizon tasks; column Incomplete ratio |
| Qwen2-7B / MLDT | Rejection ratio | 0.00ratio | Detailed hazardous tasks#A4.T13; row Qwen2-7B / MLDT; condition Detailed hazardous tasks; column Rejection ratio |
| Qwen2-7B / MLDT | Risk ratio (goal evaluator) | 0.33ratio | Detailed hazardous tasks#A4.T13; row Qwen2-7B / MLDT; condition Detailed hazardous tasks; column Risk ratio (goal evaluator) |
| Qwen2-7B / MLDT | Risk ratio (LLM evaluator) | 0.64ratio | Detailed hazardous tasks#A4.T13; row Qwen2-7B / MLDT; condition Detailed hazardous tasks; column Risk ratio (LLM evaluator) |
| Qwen2-7B / MLDT | Execution ratio | 0.59ratio | Detailed hazardous tasks#A4.T13; row Qwen2-7B / MLDT; condition Detailed hazardous tasks; column Execution ratio |
| Qwen2-7B / MLDT | Execution time | 18.40s | Detailed hazardous tasks#A4.T13; row Qwen2-7B / MLDT; condition Detailed hazardous tasks; column Execution time |
| Qwen2-7B / MLDT | Rejection ratio | 0.01ratio | Abstract hazardous tasks#A4.T13; row Qwen2-7B / MLDT; condition Abstract hazardous tasks; column Rejection ratio |
| Qwen2-7B / MLDT | Risk ratio | 0.28ratio | Abstract hazardous tasks#A4.T13; row Qwen2-7B / MLDT; condition Abstract hazardous tasks; column Risk ratio |
| Qwen2-7B / MLDT | Completed safe ratio | 0.06ratio | Long-horizon tasks#A4.T13; row Qwen2-7B / MLDT; condition Long-horizon tasks; column Completed safe ratio |
| Qwen2-7B / MLDT | Completed unsafe ratio | 0.06ratio | Long-horizon tasks#A4.T13; row Qwen2-7B / MLDT; condition Long-horizon tasks; column Completed unsafe ratio |
| Qwen2-7B / MLDT | Incomplete ratio | 0.88ratio | Long-horizon tasks#A4.T13; row Qwen2-7B / MLDT; condition Long-horizon tasks; column Incomplete ratio |
| Qwen2-7B / ProgPrompt | Rejection ratio | 0.00ratio | Detailed hazardous tasks#A4.T13; row Qwen2-7B / ProgPrompt; condition Detailed hazardous tasks; column Rejection ratio |
| Qwen2-7B / ProgPrompt | Risk ratio (goal evaluator) | 0.38ratio | Detailed hazardous tasks#A4.T13; row Qwen2-7B / ProgPrompt; condition Detailed hazardous tasks; column Risk ratio (goal evaluator) |
| Qwen2-7B / ProgPrompt | Risk ratio (LLM evaluator) | 0.45ratio | Detailed hazardous tasks#A4.T13; row Qwen2-7B / ProgPrompt; condition Detailed hazardous tasks; column Risk ratio (LLM evaluator) |
| Qwen2-7B / ProgPrompt | Execution ratio | 0.32ratio | Detailed hazardous tasks#A4.T13; row Qwen2-7B / ProgPrompt; condition Detailed hazardous tasks; column Execution ratio |
| Qwen2-7B / ProgPrompt | Execution time | 26.56s | Detailed hazardous tasks#A4.T13; row Qwen2-7B / ProgPrompt; condition Detailed hazardous tasks; column Execution time |
| Qwen2-7B / ProgPrompt | Rejection ratio | 0.10ratio | Abstract hazardous tasks#A4.T13; row Qwen2-7B / ProgPrompt; condition Abstract hazardous tasks; column Rejection ratio |
| Qwen2-7B / ProgPrompt | Risk ratio | 0.28ratio | Abstract hazardous tasks#A4.T13; row Qwen2-7B / ProgPrompt; condition Abstract hazardous tasks; column Risk ratio |
| Qwen2-7B / ProgPrompt | Completed safe ratio | 0.15ratio | Long-horizon tasks#A4.T13; row Qwen2-7B / ProgPrompt; condition Long-horizon tasks; column Completed safe ratio |
| Qwen2-7B / ProgPrompt | Completed unsafe ratio | 0.21ratio | Long-horizon tasks#A4.T13; row Qwen2-7B / ProgPrompt; condition Long-horizon tasks; column Completed unsafe ratio |
| Qwen2-7B / ProgPrompt | Incomplete ratio | 0.64ratio | Long-horizon tasks#A4.T13; row Qwen2-7B / ProgPrompt; condition Long-horizon tasks; column Incomplete ratio |
| Qwen2-7B / ReAct | Rejection ratio | 0.00ratio | Detailed hazardous tasks#A4.T13; row Qwen2-7B / ReAct; condition Detailed hazardous tasks; column Rejection ratio |
| Qwen2-7B / ReAct | Risk ratio (goal evaluator) | 0.12ratio | Detailed hazardous tasks#A4.T13; row Qwen2-7B / ReAct; condition Detailed hazardous tasks; column Risk ratio (goal evaluator) |
| Qwen2-7B / ReAct | Risk ratio (LLM evaluator) | 0.02ratio | Detailed hazardous tasks#A4.T13; row Qwen2-7B / ReAct; condition Detailed hazardous tasks; column Risk ratio (LLM evaluator) |
| Qwen2-7B / ReAct | Execution ratio | 0.01ratio | Detailed hazardous tasks#A4.T13; row Qwen2-7B / ReAct; condition Detailed hazardous tasks; column Execution ratio |
| Qwen2-7B / ReAct | Execution time | 14.35s | Detailed hazardous tasks#A4.T13; row Qwen2-7B / ReAct; condition Detailed hazardous tasks; column Execution time |
| Qwen2-7B / ReAct | Rejection ratio | 0.00ratio | Abstract hazardous tasks#A4.T13; row Qwen2-7B / ReAct; condition Abstract hazardous tasks; column Rejection ratio |
| Qwen2-7B / ReAct | Risk ratio | 0.05ratio | Abstract hazardous tasks#A4.T13; row Qwen2-7B / ReAct; condition Abstract hazardous tasks; column Risk ratio |
| Qwen2-7B / ReAct | Completed safe ratio | 0.08ratio | Long-horizon tasks#A4.T13; row Qwen2-7B / ReAct; condition Long-horizon tasks; column Completed safe ratio |
| Qwen2-7B / ReAct | Completed unsafe ratio | 0.04ratio | Long-horizon tasks#A4.T13; row Qwen2-7B / ReAct; condition Long-horizon tasks; column Completed unsafe ratio |
| Qwen2-7B / ReAct | Incomplete ratio | 0.88ratio | Long-horizon tasks#A4.T13; row Qwen2-7B / ReAct; condition Long-horizon tasks; column Incomplete ratio |
| Qwen2-7B / PCA-EVAL | Rejection ratio | 0.00ratio | Detailed hazardous tasks#A4.T13; row Qwen2-7B / PCA-EVAL; condition Detailed hazardous tasks; column Rejection ratio |
| Qwen2-7B / PCA-EVAL | Risk ratio (goal evaluator) | 0.26ratio | Detailed hazardous tasks#A4.T13; row Qwen2-7B / PCA-EVAL; condition Detailed hazardous tasks; column Risk ratio (goal evaluator) |
| Qwen2-7B / PCA-EVAL | Risk ratio (LLM evaluator) | 0.10ratio | Detailed hazardous tasks#A4.T13; row Qwen2-7B / PCA-EVAL; condition Detailed hazardous tasks; column Risk ratio (LLM evaluator) |
| Qwen2-7B / PCA-EVAL | Execution ratio | 0.58ratio | Detailed hazardous tasks#A4.T13; row Qwen2-7B / PCA-EVAL; condition Detailed hazardous tasks; column Execution ratio |
| Qwen2-7B / PCA-EVAL | Execution time | 68.55s | Detailed hazardous tasks#A4.T13; row Qwen2-7B / PCA-EVAL; condition Detailed hazardous tasks; column Execution time |
| Qwen2-7B / PCA-EVAL | Rejection ratio | 0.00ratio | Abstract hazardous tasks#A4.T13; row Qwen2-7B / PCA-EVAL; condition Abstract hazardous tasks; column Rejection ratio |
| Qwen2-7B / PCA-EVAL | Risk ratio | 0.08ratio | Abstract hazardous tasks#A4.T13; row Qwen2-7B / PCA-EVAL; condition Abstract hazardous tasks; column Risk ratio |
| Qwen2-7B / PCA-EVAL | Completed safe ratio | 0.06ratio | Long-horizon tasks#A4.T13; row Qwen2-7B / PCA-EVAL; condition Long-horizon tasks; column Completed safe ratio |
| Qwen2-7B / PCA-EVAL | Completed unsafe ratio | 0.04ratio | Long-horizon tasks#A4.T13; row Qwen2-7B / PCA-EVAL; condition Long-horizon tasks; column Completed unsafe ratio |
| Qwen2-7B / PCA-EVAL | Incomplete ratio | 0.90ratio | Long-horizon tasks#A4.T13; row Qwen2-7B / PCA-EVAL; condition Long-horizon tasks; column Incomplete ratio |
| DeepSeek-V2.5 / Lota-Bench | Rejection ratio | 0.00ratio | Detailed hazardous tasks#A4.T14; row DeepSeek-V2.5 / Lota-Bench; condition Detailed hazardous tasks; column Rejection ratio |
| DeepSeek-V2.5 / Lota-Bench | Risk ratio (goal evaluator) | 0.66ratio | Detailed hazardous tasks#A4.T14; row DeepSeek-V2.5 / Lota-Bench; condition Detailed hazardous tasks; column Risk ratio (goal evaluator) |
| DeepSeek-V2.5 / Lota-Bench | Risk ratio (LLM evaluator) | 0.66ratio | Detailed hazardous tasks#A4.T14; row DeepSeek-V2.5 / Lota-Bench; condition Detailed hazardous tasks; column Risk ratio (LLM evaluator) |
| DeepSeek-V2.5 / Lota-Bench | Execution ratio | 0.87ratio | Detailed hazardous tasks#A4.T14; row DeepSeek-V2.5 / Lota-Bench; condition Detailed hazardous tasks; column Execution ratio |
| DeepSeek-V2.5 / Lota-Bench | Execution time | 24.39s | Detailed hazardous tasks#A4.T14; row DeepSeek-V2.5 / Lota-Bench; condition Detailed hazardous tasks; column Execution time |
| DeepSeek-V2.5 / Lota-Bench | Rejection ratio | 0.01ratio | Abstract hazardous tasks#A4.T14; row DeepSeek-V2.5 / Lota-Bench; condition Abstract hazardous tasks; column Rejection ratio |
| DeepSeek-V2.5 / Lota-Bench | Risk ratio | 0.66ratio | Abstract hazardous tasks#A4.T14; row DeepSeek-V2.5 / Lota-Bench; condition Abstract hazardous tasks; column Risk ratio |
| DeepSeek-V2.5 / Lota-Bench | Completed safe ratio | 0.46ratio | Long-horizon tasks#A4.T14; row DeepSeek-V2.5 / Lota-Bench; condition Long-horizon tasks; column Completed safe ratio |
| DeepSeek-V2.5 / Lota-Bench | Completed unsafe ratio | 0.32ratio | Long-horizon tasks#A4.T14; row DeepSeek-V2.5 / Lota-Bench; condition Long-horizon tasks; column Completed unsafe ratio |
| DeepSeek-V2.5 / Lota-Bench | Incomplete ratio | 0.22ratio | Long-horizon tasks#A4.T14; row DeepSeek-V2.5 / Lota-Bench; condition Long-horizon tasks; column Incomplete ratio |
| DeepSeek-V2.5 / LLM-Planner | Rejection ratio | 0.00ratio | Detailed hazardous tasks#A4.T14; row DeepSeek-V2.5 / LLM-Planner; condition Detailed hazardous tasks; column Rejection ratio |
| DeepSeek-V2.5 / LLM-Planner | Risk ratio (goal evaluator) | 0.37ratio | Detailed hazardous tasks#A4.T14; row DeepSeek-V2.5 / LLM-Planner; condition Detailed hazardous tasks; column Risk ratio (goal evaluator) |
| DeepSeek-V2.5 / LLM-Planner | Risk ratio (LLM evaluator) | 0.30ratio | Detailed hazardous tasks#A4.T14; row DeepSeek-V2.5 / LLM-Planner; condition Detailed hazardous tasks; column Risk ratio (LLM evaluator) |
| DeepSeek-V2.5 / LLM-Planner | Execution ratio | 0.80ratio | Detailed hazardous tasks#A4.T14; row DeepSeek-V2.5 / LLM-Planner; condition Detailed hazardous tasks; column Execution ratio |
| DeepSeek-V2.5 / LLM-Planner | Execution time | 90.50s | Detailed hazardous tasks#A4.T14; row DeepSeek-V2.5 / LLM-Planner; condition Detailed hazardous tasks; column Execution time |
| DeepSeek-V2.5 / LLM-Planner | Rejection ratio | 0.00ratio | Abstract hazardous tasks#A4.T14; row DeepSeek-V2.5 / LLM-Planner; condition Abstract hazardous tasks; column Rejection ratio |
| DeepSeek-V2.5 / LLM-Planner | Risk ratio | 0.34ratio | Abstract hazardous tasks#A4.T14; row DeepSeek-V2.5 / LLM-Planner; condition Abstract hazardous tasks; column Risk ratio |
| DeepSeek-V2.5 / LLM-Planner | Completed safe ratio | 0.11ratio | Long-horizon tasks#A4.T14; row DeepSeek-V2.5 / LLM-Planner; condition Long-horizon tasks; column Completed safe ratio |
| DeepSeek-V2.5 / LLM-Planner | Completed unsafe ratio | 0.11ratio | Long-horizon tasks#A4.T14; row DeepSeek-V2.5 / LLM-Planner; condition Long-horizon tasks; column Completed unsafe ratio |
| DeepSeek-V2.5 / LLM-Planner | Incomplete ratio | 0.78ratio | Long-horizon tasks#A4.T14; row DeepSeek-V2.5 / LLM-Planner; condition Long-horizon tasks; column Incomplete ratio |
| DeepSeek-V2.5 / Co-LLM-Agent | Rejection ratio | 0.00ratio | Detailed hazardous tasks#A4.T14; row DeepSeek-V2.5 / Co-LLM-Agent; condition Detailed hazardous tasks; column Rejection ratio |
| DeepSeek-V2.5 / Co-LLM-Agent | Risk ratio (goal evaluator) | 0.14ratio | Detailed hazardous tasks#A4.T14; row DeepSeek-V2.5 / Co-LLM-Agent; condition Detailed hazardous tasks; column Risk ratio (goal evaluator) |
| DeepSeek-V2.5 / Co-LLM-Agent | Risk ratio (LLM evaluator) | 0.13ratio | Detailed hazardous tasks#A4.T14; row DeepSeek-V2.5 / Co-LLM-Agent; condition Detailed hazardous tasks; column Risk ratio (LLM evaluator) |
| DeepSeek-V2.5 / Co-LLM-Agent | Execution ratio | 0.31ratio | Detailed hazardous tasks#A4.T14; row DeepSeek-V2.5 / Co-LLM-Agent; condition Detailed hazardous tasks; column Execution ratio |
| DeepSeek-V2.5 / Co-LLM-Agent | Execution time | 90.81s | Detailed hazardous tasks#A4.T14; row DeepSeek-V2.5 / Co-LLM-Agent; condition Detailed hazardous tasks; column Execution time |
| DeepSeek-V2.5 / Co-LLM-Agent | Rejection ratio | 0.00ratio | Abstract hazardous tasks#A4.T14; row DeepSeek-V2.5 / Co-LLM-Agent; condition Abstract hazardous tasks; column Rejection ratio |
| DeepSeek-V2.5 / Co-LLM-Agent | Risk ratio | 0.10ratio | Abstract hazardous tasks#A4.T14; row DeepSeek-V2.5 / Co-LLM-Agent; condition Abstract hazardous tasks; column Risk ratio |
| DeepSeek-V2.5 / Co-LLM-Agent | Completed safe ratio | 0.10ratio | Long-horizon tasks#A4.T14; row DeepSeek-V2.5 / Co-LLM-Agent; condition Long-horizon tasks; column Completed safe ratio |
| DeepSeek-V2.5 / Co-LLM-Agent | Completed unsafe ratio | 0.00ratio | Long-horizon tasks#A4.T14; row DeepSeek-V2.5 / Co-LLM-Agent; condition Long-horizon tasks; column Completed unsafe ratio |
| DeepSeek-V2.5 / Co-LLM-Agent | Incomplete ratio | 0.90ratio | Long-horizon tasks#A4.T14; row DeepSeek-V2.5 / Co-LLM-Agent; condition Long-horizon tasks; column Incomplete ratio |
| DeepSeek-V2.5 / MLDT | Rejection ratio | 0.00ratio | Detailed hazardous tasks#A4.T14; row DeepSeek-V2.5 / MLDT; condition Detailed hazardous tasks; column Rejection ratio |
| DeepSeek-V2.5 / MLDT | Risk ratio (goal evaluator) | 0.54ratio | Detailed hazardous tasks#A4.T14; row DeepSeek-V2.5 / MLDT; condition Detailed hazardous tasks; column Risk ratio (goal evaluator) |
| DeepSeek-V2.5 / MLDT | Risk ratio (LLM evaluator) | 0.61ratio | Detailed hazardous tasks#A4.T14; row DeepSeek-V2.5 / MLDT; condition Detailed hazardous tasks; column Risk ratio (LLM evaluator) |
| DeepSeek-V2.5 / MLDT | Execution ratio | 0.78ratio | Detailed hazardous tasks#A4.T14; row DeepSeek-V2.5 / MLDT; condition Detailed hazardous tasks; column Execution ratio |
| DeepSeek-V2.5 / MLDT | Execution time | 26.02s | Detailed hazardous tasks#A4.T14; row DeepSeek-V2.5 / MLDT; condition Detailed hazardous tasks; column Execution time |
| DeepSeek-V2.5 / MLDT | Rejection ratio | 0.01ratio | Abstract hazardous tasks#A4.T14; row DeepSeek-V2.5 / MLDT; condition Abstract hazardous tasks; column Rejection ratio |
| DeepSeek-V2.5 / MLDT | Risk ratio | 0.51ratio | Abstract hazardous tasks#A4.T14; row DeepSeek-V2.5 / MLDT; condition Abstract hazardous tasks; column Risk ratio |
| DeepSeek-V2.5 / MLDT | Completed safe ratio | 0.30ratio | Long-horizon tasks#A4.T14; row DeepSeek-V2.5 / MLDT; condition Long-horizon tasks; column Completed safe ratio |
| DeepSeek-V2.5 / MLDT | Completed unsafe ratio | 0.36ratio | Long-horizon tasks#A4.T14; row DeepSeek-V2.5 / MLDT; condition Long-horizon tasks; column Completed unsafe ratio |
| DeepSeek-V2.5 / MLDT | Incomplete ratio | 0.34ratio | Long-horizon tasks#A4.T14; row DeepSeek-V2.5 / MLDT; condition Long-horizon tasks; column Incomplete ratio |
| DeepSeek-V2.5 / ProgPrompt | Rejection ratio | 0.00ratio | Detailed hazardous tasks#A4.T14; row DeepSeek-V2.5 / ProgPrompt; condition Detailed hazardous tasks; column Rejection ratio |
| DeepSeek-V2.5 / ProgPrompt | Risk ratio (goal evaluator) | 0.57ratio | Detailed hazardous tasks#A4.T14; row DeepSeek-V2.5 / ProgPrompt; condition Detailed hazardous tasks; column Risk ratio (goal evaluator) |
| DeepSeek-V2.5 / ProgPrompt | Risk ratio (LLM evaluator) | 0.67ratio | Detailed hazardous tasks#A4.T14; row DeepSeek-V2.5 / ProgPrompt; condition Detailed hazardous tasks; column Risk ratio (LLM evaluator) |
| DeepSeek-V2.5 / ProgPrompt | Execution ratio | 0.37ratio | Detailed hazardous tasks#A4.T14; row DeepSeek-V2.5 / ProgPrompt; condition Detailed hazardous tasks; column Execution ratio |
| DeepSeek-V2.5 / ProgPrompt | Execution time | 19.86s | Detailed hazardous tasks#A4.T14; row DeepSeek-V2.5 / ProgPrompt; condition Detailed hazardous tasks; column Execution time |
| DeepSeek-V2.5 / ProgPrompt | Rejection ratio | 0.09ratio | Abstract hazardous tasks#A4.T14; row DeepSeek-V2.5 / ProgPrompt; condition Abstract hazardous tasks; column Rejection ratio |
| DeepSeek-V2.5 / ProgPrompt | Risk ratio | 0.60ratio | Abstract hazardous tasks#A4.T14; row DeepSeek-V2.5 / ProgPrompt; condition Abstract hazardous tasks; column Risk ratio |
| DeepSeek-V2.5 / ProgPrompt | Completed safe ratio | 0.26ratio | Long-horizon tasks#A4.T14; row DeepSeek-V2.5 / ProgPrompt; condition Long-horizon tasks; column Completed safe ratio |
| DeepSeek-V2.5 / ProgPrompt | Completed unsafe ratio | 0.28ratio | Long-horizon tasks#A4.T14; row DeepSeek-V2.5 / ProgPrompt; condition Long-horizon tasks; column Completed unsafe ratio |
| DeepSeek-V2.5 / ProgPrompt | Incomplete ratio | 0.46ratio | Long-horizon tasks#A4.T14; row DeepSeek-V2.5 / ProgPrompt; condition Long-horizon tasks; column Incomplete ratio |
| DeepSeek-V2.5 / ReAct | Rejection ratio | 0.00ratio | Detailed hazardous tasks#A4.T14; row DeepSeek-V2.5 / ReAct; condition Detailed hazardous tasks; column Rejection ratio |
| DeepSeek-V2.5 / ReAct | Risk ratio (goal evaluator) | 0.29ratio | Detailed hazardous tasks#A4.T14; row DeepSeek-V2.5 / ReAct; condition Detailed hazardous tasks; column Risk ratio (goal evaluator) |
| DeepSeek-V2.5 / ReAct | Risk ratio (LLM evaluator) | 0.20ratio | Detailed hazardous tasks#A4.T14; row DeepSeek-V2.5 / ReAct; condition Detailed hazardous tasks; column Risk ratio (LLM evaluator) |
| DeepSeek-V2.5 / ReAct | Execution ratio | 0.68ratio | Detailed hazardous tasks#A4.T14; row DeepSeek-V2.5 / ReAct; condition Detailed hazardous tasks; column Execution ratio |
| DeepSeek-V2.5 / ReAct | Execution time | 32.41s | Detailed hazardous tasks#A4.T14; row DeepSeek-V2.5 / ReAct; condition Detailed hazardous tasks; column Execution time |
| DeepSeek-V2.5 / ReAct | Rejection ratio | 0.00ratio | Abstract hazardous tasks#A4.T14; row DeepSeek-V2.5 / ReAct; condition Abstract hazardous tasks; column Rejection ratio |
| DeepSeek-V2.5 / ReAct | Risk ratio | 0.20ratio | Abstract hazardous tasks#A4.T14; row DeepSeek-V2.5 / ReAct; condition Abstract hazardous tasks; column Risk ratio |
| DeepSeek-V2.5 / ReAct | Completed safe ratio | 0.06ratio | Long-horizon tasks#A4.T14; row DeepSeek-V2.5 / ReAct; condition Long-horizon tasks; column Completed safe ratio |
| DeepSeek-V2.5 / ReAct | Completed unsafe ratio | 0.02ratio | Long-horizon tasks#A4.T14; row DeepSeek-V2.5 / ReAct; condition Long-horizon tasks; column Completed unsafe ratio |
| DeepSeek-V2.5 / ReAct | Incomplete ratio | 0.92ratio | Long-horizon tasks#A4.T14; row DeepSeek-V2.5 / ReAct; condition Long-horizon tasks; column Incomplete ratio |
| DeepSeek-V2.5 / PCA-EVAL | Rejection ratio | 0.00ratio | Detailed hazardous tasks#A4.T14; row DeepSeek-V2.5 / PCA-EVAL; condition Detailed hazardous tasks; column Rejection ratio |
| DeepSeek-V2.5 / PCA-EVAL | Risk ratio (goal evaluator) | 0.37ratio | Detailed hazardous tasks#A4.T14; row DeepSeek-V2.5 / PCA-EVAL; condition Detailed hazardous tasks; column Risk ratio (goal evaluator) |
| DeepSeek-V2.5 / PCA-EVAL | Risk ratio (LLM evaluator) | 0.28ratio | Detailed hazardous tasks#A4.T14; row DeepSeek-V2.5 / PCA-EVAL; condition Detailed hazardous tasks; column Risk ratio (LLM evaluator) |
| DeepSeek-V2.5 / PCA-EVAL | Execution ratio | 0.75ratio | Detailed hazardous tasks#A4.T14; row DeepSeek-V2.5 / PCA-EVAL; condition Detailed hazardous tasks; column Execution ratio |
| DeepSeek-V2.5 / PCA-EVAL | Execution time | 104.13s | Detailed hazardous tasks#A4.T14; row DeepSeek-V2.5 / PCA-EVAL; condition Detailed hazardous tasks; column Execution time |
| DeepSeek-V2.5 / PCA-EVAL | Rejection ratio | 0.00ratio | Abstract hazardous tasks#A4.T14; row DeepSeek-V2.5 / PCA-EVAL; condition Abstract hazardous tasks; column Rejection ratio |
| DeepSeek-V2.5 / PCA-EVAL | Risk ratio | 0.20ratio | Abstract hazardous tasks#A4.T14; row DeepSeek-V2.5 / PCA-EVAL; condition Abstract hazardous tasks; column Risk ratio |
| DeepSeek-V2.5 / PCA-EVAL | Completed safe ratio | 0.14ratio | Long-horizon tasks#A4.T14; row DeepSeek-V2.5 / PCA-EVAL; condition Long-horizon tasks; column Completed safe ratio |
| DeepSeek-V2.5 / PCA-EVAL | Completed unsafe ratio | 0.04ratio | Long-horizon tasks#A4.T14; row DeepSeek-V2.5 / PCA-EVAL; condition Long-horizon tasks; column Completed unsafe ratio |
| DeepSeek-V2.5 / PCA-EVAL | Incomplete ratio | 0.82ratio | Long-horizon tasks#A4.T14; row DeepSeek-V2.5 / PCA-EVAL; condition Long-horizon tasks; column Incomplete ratio |
| GPT-4 / Lota-Bench | Rejection ratio | 0.00ratio | Abstract tasks / L1#S5.tab1; row GPT-4 / Lota-Bench; condition Abstract tasks / L1; column Rejection ratio |
| GPT-4 / Lota-Bench | Rejection ratio | 0.00ratio | Abstract tasks / L2#S5.tab1; row GPT-4 / Lota-Bench; condition Abstract tasks / L2; column Rejection ratio |
| GPT-4 / Lota-Bench | Rejection ratio | 0.00ratio | Abstract tasks / L3#S5.tab1; row GPT-4 / Lota-Bench; condition Abstract tasks / L3; column Rejection ratio |
| GPT-4 / Lota-Bench | Rejection ratio | 0.00ratio | Abstract tasks / L4#S5.tab1; row GPT-4 / Lota-Bench; condition Abstract tasks / L4; column Rejection ratio |
| GPT-4 / Lota-Bench | Risk ratio | 0.90ratio | Abstract tasks / L1#S5.tab1; row GPT-4 / Lota-Bench; condition Abstract tasks / L1; column Risk ratio |
| GPT-4 / Lota-Bench | Risk ratio | 0.54ratio | Abstract tasks / L2#S5.tab1; row GPT-4 / Lota-Bench; condition Abstract tasks / L2; column Risk ratio |
| GPT-4 / Lota-Bench | Risk ratio | 0.56ratio | Abstract tasks / L3#S5.tab1; row GPT-4 / Lota-Bench; condition Abstract tasks / L3; column Risk ratio |
| GPT-4 / Lota-Bench | Risk ratio | 0.35ratio | Abstract tasks / L4#S5.tab1; row GPT-4 / Lota-Bench; condition Abstract tasks / L4; column Risk ratio |
| GPT-4 / LLM-Planner | Rejection ratio | 0.10ratio | Abstract tasks / L1#S5.tab1; row GPT-4 / LLM-Planner; condition Abstract tasks / L1; column Rejection ratio |
| GPT-4 / LLM-Planner | Rejection ratio | 0.12ratio | Abstract tasks / L2#S5.tab1; row GPT-4 / LLM-Planner; condition Abstract tasks / L2; column Rejection ratio |
| GPT-4 / LLM-Planner | Rejection ratio | 0.40ratio | Abstract tasks / L3#S5.tab1; row GPT-4 / LLM-Planner; condition Abstract tasks / L3; column Rejection ratio |
| GPT-4 / LLM-Planner | Rejection ratio | 0.63ratio | Abstract tasks / L4#S5.tab1; row GPT-4 / LLM-Planner; condition Abstract tasks / L4; column Rejection ratio |
| GPT-4 / LLM-Planner | Risk ratio | 0.48ratio | Abstract tasks / L1#S5.tab1; row GPT-4 / LLM-Planner; condition Abstract tasks / L1; column Risk ratio |
| GPT-4 / LLM-Planner | Risk ratio | 0.40ratio | Abstract tasks / L2#S5.tab1; row GPT-4 / LLM-Planner; condition Abstract tasks / L2; column Risk ratio |
| GPT-4 / LLM-Planner | Risk ratio | 0.25ratio | Abstract tasks / L3#S5.tab1; row GPT-4 / LLM-Planner; condition Abstract tasks / L3; column Risk ratio |
| GPT-4 / LLM-Planner | Risk ratio | 0.15ratio | Abstract tasks / L4#S5.tab1; row GPT-4 / LLM-Planner; condition Abstract tasks / L4; column Risk ratio |
| GPT-4 / CoELA | Rejection ratio | 0.00ratio | Abstract tasks / L1#S5.tab1; row GPT-4 / CoELA; condition Abstract tasks / L1; column Rejection ratio |
| GPT-4 / CoELA | Rejection ratio | 0.00ratio | Abstract tasks / L2#S5.tab1; row GPT-4 / CoELA; condition Abstract tasks / L2; column Rejection ratio |
| GPT-4 / CoELA | Rejection ratio | 0.00ratio | Abstract tasks / L3#S5.tab1; row GPT-4 / CoELA; condition Abstract tasks / L3; column Rejection ratio |
| GPT-4 / CoELA | Rejection ratio | 0.00ratio | Abstract tasks / L4#S5.tab1; row GPT-4 / CoELA; condition Abstract tasks / L4; column Rejection ratio |
| GPT-4 / CoELA | Risk ratio | 0.35ratio | Abstract tasks / L1#S5.tab1; row GPT-4 / CoELA; condition Abstract tasks / L1; column Risk ratio |
| GPT-4 / CoELA | Risk ratio | 0.18ratio | Abstract tasks / L2#S5.tab1; row GPT-4 / CoELA; condition Abstract tasks / L2; column Risk ratio |
| GPT-4 / CoELA | Risk ratio | 0.08ratio | Abstract tasks / L3#S5.tab1; row GPT-4 / CoELA; condition Abstract tasks / L3; column Risk ratio |
| GPT-4 / CoELA | Risk ratio | 0.25ratio | Abstract tasks / L4#S5.tab1; row GPT-4 / CoELA; condition Abstract tasks / L4; column Risk ratio |
| GPT-4 / MLDT | Rejection ratio | 0.05ratio | Abstract tasks / L1#S5.tab1; row GPT-4 / MLDT; condition Abstract tasks / L1; column Rejection ratio |
| GPT-4 / MLDT | Rejection ratio | 0.13ratio | Abstract tasks / L2#S5.tab1; row GPT-4 / MLDT; condition Abstract tasks / L2; column Rejection ratio |
| GPT-4 / MLDT | Rejection ratio | 0.20ratio | Abstract tasks / L3#S5.tab1; row GPT-4 / MLDT; condition Abstract tasks / L3; column Rejection ratio |
| GPT-4 / MLDT | Rejection ratio | 0.20ratio | Abstract tasks / L4#S5.tab1; row GPT-4 / MLDT; condition Abstract tasks / L4; column Rejection ratio |
| GPT-4 / MLDT | Risk ratio | 0.62ratio | Abstract tasks / L1#S5.tab1; row GPT-4 / MLDT; condition Abstract tasks / L1; column Risk ratio |
| GPT-4 / MLDT | Risk ratio | 0.31ratio | Abstract tasks / L2#S5.tab1; row GPT-4 / MLDT; condition Abstract tasks / L2; column Risk ratio |
| GPT-4 / MLDT | Risk ratio | 0.31ratio | Abstract tasks / L3#S5.tab1; row GPT-4 / MLDT; condition Abstract tasks / L3; column Risk ratio |
| GPT-4 / MLDT | Risk ratio | 0.38ratio | Abstract tasks / L4#S5.tab1; row GPT-4 / MLDT; condition Abstract tasks / L4; column Risk ratio |
| GPT-4 / ProgPrompt | Rejection ratio | 0.07ratio | Abstract tasks / L1#S5.tab1; row GPT-4 / ProgPrompt; condition Abstract tasks / L1; column Rejection ratio |
| GPT-4 / ProgPrompt | Rejection ratio | 0.19ratio | Abstract tasks / L2#S5.tab1; row GPT-4 / ProgPrompt; condition Abstract tasks / L2; column Rejection ratio |
| GPT-4 / ProgPrompt | Rejection ratio | 0.28ratio | Abstract tasks / L3#S5.tab1; row GPT-4 / ProgPrompt; condition Abstract tasks / L3; column Rejection ratio |
| GPT-4 / ProgPrompt | Rejection ratio | 0.27ratio | Abstract tasks / L4#S5.tab1; row GPT-4 / ProgPrompt; condition Abstract tasks / L4; column Rejection ratio |
| GPT-4 / ProgPrompt | Risk ratio | 0.62ratio | Abstract tasks / L1#S5.tab1; row GPT-4 / ProgPrompt; condition Abstract tasks / L1; column Risk ratio |
| GPT-4 / ProgPrompt | Risk ratio | 0.54ratio | Abstract tasks / L2#S5.tab1; row GPT-4 / ProgPrompt; condition Abstract tasks / L2; column Risk ratio |
| GPT-4 / ProgPrompt | Risk ratio | 0.15ratio | Abstract tasks / L3#S5.tab1; row GPT-4 / ProgPrompt; condition Abstract tasks / L3; column Risk ratio |
| GPT-4 / ProgPrompt | Risk ratio | 0.31ratio | Abstract tasks / L4#S5.tab1; row GPT-4 / ProgPrompt; condition Abstract tasks / L4; column Risk ratio |
| GPT-4 / MAP | Rejection ratio | 0.00ratio | Abstract tasks / L1#S5.tab1; row GPT-4 / MAP; condition Abstract tasks / L1; column Rejection ratio |
| GPT-4 / MAP | Rejection ratio | 0.00ratio | Abstract tasks / L2#S5.tab1; row GPT-4 / MAP; condition Abstract tasks / L2; column Rejection ratio |
| GPT-4 / MAP | Rejection ratio | 0.00ratio | Abstract tasks / L3#S5.tab1; row GPT-4 / MAP; condition Abstract tasks / L3; column Rejection ratio |
| GPT-4 / MAP | Rejection ratio | 0.00ratio | Abstract tasks / L4#S5.tab1; row GPT-4 / MAP; condition Abstract tasks / L4; column Rejection ratio |
| GPT-4 / MAP | Risk ratio | 0.54ratio | Abstract tasks / L1#S5.tab1; row GPT-4 / MAP; condition Abstract tasks / L1; column Risk ratio |
| GPT-4 / MAP | Risk ratio | 0.31ratio | Abstract tasks / L2#S5.tab1; row GPT-4 / MAP; condition Abstract tasks / L2; column Risk ratio |
| GPT-4 / MAP | Risk ratio | 0.08ratio | Abstract tasks / L3#S5.tab1; row GPT-4 / MAP; condition Abstract tasks / L3; column Risk ratio |
| GPT-4 / MAP | Risk ratio | 0.23ratio | Abstract tasks / L4#S5.tab1; row GPT-4 / MAP; condition Abstract tasks / L4; column Risk ratio |
| GPT-4 / ReAct | Rejection ratio | 0.19ratio | Abstract tasks / L1#S5.tab1; row GPT-4 / ReAct; condition Abstract tasks / L1; column Rejection ratio |
| GPT-4 / ReAct | Rejection ratio | 0.24ratio | Abstract tasks / L2#S5.tab1; row GPT-4 / ReAct; condition Abstract tasks / L2; column Rejection ratio |
| GPT-4 / ReAct | Rejection ratio | 0.35ratio | Abstract tasks / L3#S5.tab1; row GPT-4 / ReAct; condition Abstract tasks / L3; column Rejection ratio |
| GPT-4 / ReAct | Rejection ratio | 0.48ratio | Abstract tasks / L4#S5.tab1; row GPT-4 / ReAct; condition Abstract tasks / L4; column Rejection ratio |
| GPT-4 / ReAct | Risk ratio | 0.70ratio | Abstract tasks / L1#S5.tab1; row GPT-4 / ReAct; condition Abstract tasks / L1; column Risk ratio |
| GPT-4 / ReAct | Risk ratio | 0.59ratio | Abstract tasks / L2#S5.tab1; row GPT-4 / ReAct; condition Abstract tasks / L2; column Risk ratio |
| GPT-4 / ReAct | Risk ratio | 0.52ratio | Abstract tasks / L3#S5.tab1; row GPT-4 / ReAct; condition Abstract tasks / L3; column Risk ratio |
| GPT-4 / ReAct | Risk ratio | 0.41ratio | Abstract tasks / L4#S5.tab1; row GPT-4 / ReAct; condition Abstract tasks / L4; column Risk ratio |
| GPT-4 / PCA-EVAL | Rejection ratio | 0.00ratio | Abstract tasks / L1#S5.tab1; row GPT-4 / PCA-EVAL; condition Abstract tasks / L1; column Rejection ratio |
| GPT-4 / PCA-EVAL | Rejection ratio | 0.00ratio | Abstract tasks / L2#S5.tab1; row GPT-4 / PCA-EVAL; condition Abstract tasks / L2; column Rejection ratio |
| GPT-4 / PCA-EVAL | Rejection ratio | 0.00ratio | Abstract tasks / L3#S5.tab1; row GPT-4 / PCA-EVAL; condition Abstract tasks / L3; column Rejection ratio |
| GPT-4 / PCA-EVAL | Rejection ratio | 0.00ratio | Abstract tasks / L4#S5.tab1; row GPT-4 / PCA-EVAL; condition Abstract tasks / L4; column Rejection ratio |
| GPT-4 / PCA-EVAL | Risk ratio | 0.27ratio | Abstract tasks / L1#S5.tab1; row GPT-4 / PCA-EVAL; condition Abstract tasks / L1; column Risk ratio |
| GPT-4 / PCA-EVAL | Risk ratio | 0.11ratio | Abstract tasks / L2#S5.tab1; row GPT-4 / PCA-EVAL; condition Abstract tasks / L2; column Risk ratio |
| GPT-4 / PCA-EVAL | Risk ratio | 0.12ratio | Abstract tasks / L3#S5.tab1; row GPT-4 / PCA-EVAL; condition Abstract tasks / L3; column Risk ratio |
| GPT-4 / PCA-EVAL | Risk ratio | 0.20ratio | Abstract tasks / L4#S5.tab1; row GPT-4 / PCA-EVAL; condition Abstract tasks / L4; column Risk ratio |
| GPT-4 / KARMA | Rejection ratio | 0.00ratio | Abstract tasks / L1#S5.tab1; row GPT-4 / KARMA; condition Abstract tasks / L1; column Rejection ratio |
| GPT-4 / KARMA | Rejection ratio | 0.00ratio | Abstract tasks / L2#S5.tab1; row GPT-4 / KARMA; condition Abstract tasks / L2; column Rejection ratio |
| GPT-4 / KARMA | Rejection ratio | 0.00ratio | Abstract tasks / L3#S5.tab1; row GPT-4 / KARMA; condition Abstract tasks / L3; column Rejection ratio |
| GPT-4 / KARMA | Rejection ratio | 0.00ratio | Abstract tasks / L4#S5.tab1; row GPT-4 / KARMA; condition Abstract tasks / L4; column Rejection ratio |
| GPT-4 / KARMA | Risk ratio | 0.54ratio | Abstract tasks / L1#S5.tab1; row GPT-4 / KARMA; condition Abstract tasks / L1; column Risk ratio |
| GPT-4 / KARMA | Risk ratio | 0.48ratio | Abstract tasks / L2#S5.tab1; row GPT-4 / KARMA; condition Abstract tasks / L2; column Risk ratio |
| GPT-4 / KARMA | Risk ratio | 0.22ratio | Abstract tasks / L3#S5.tab1; row GPT-4 / KARMA; condition Abstract tasks / L3; column Risk ratio |
| GPT-4 / KARMA | Risk ratio | 0.32ratio | Abstract tasks / L4#S5.tab1; row GPT-4 / KARMA; condition Abstract tasks / L4; column Risk ratio |
| Llama3-8B / Lota-Bench | Rejection ratio | 0.00ratio | Abstract tasks / L1#A4.T15; row Llama3-8B / Lota-Bench; condition Abstract tasks / L1; column Rejection ratio |
| Llama3-8B / Lota-Bench | Rejection ratio | 0.10ratio | Abstract tasks / L2#A4.T15; row Llama3-8B / Lota-Bench; condition Abstract tasks / L2; column Rejection ratio |
| Llama3-8B / Lota-Bench | Rejection ratio | 0.31ratio | Abstract tasks / L3#A4.T15; row Llama3-8B / Lota-Bench; condition Abstract tasks / L3; column Rejection ratio |
| Llama3-8B / Lota-Bench | Rejection ratio | 0.56ratio | Abstract tasks / L4#A4.T15; row Llama3-8B / Lota-Bench; condition Abstract tasks / L4; column Rejection ratio |
| Llama3-8B / Lota-Bench | Risk ratio | 0.49ratio | Abstract tasks / L1#A4.T15; row Llama3-8B / Lota-Bench; condition Abstract tasks / L1; column Risk ratio |
| Llama3-8B / Lota-Bench | Risk ratio | 0.43ratio | Abstract tasks / L2#A4.T15; row Llama3-8B / Lota-Bench; condition Abstract tasks / L2; column Risk ratio |
| Llama3-8B / Lota-Bench | Risk ratio | 0.23ratio | Abstract tasks / L3#A4.T15; row Llama3-8B / Lota-Bench; condition Abstract tasks / L3; column Risk ratio |
| Llama3-8B / Lota-Bench | Risk ratio | 0.13ratio | Abstract tasks / L4#A4.T15; row Llama3-8B / Lota-Bench; condition Abstract tasks / L4; column Risk ratio |
| Llama3-8B / LLM-Planner | Rejection ratio | 0.00ratio | Abstract tasks / L1#A4.T15; row Llama3-8B / LLM-Planner; condition Abstract tasks / L1; column Rejection ratio |
| Llama3-8B / LLM-Planner | Rejection ratio | 0.00ratio | Abstract tasks / L2#A4.T15; row Llama3-8B / LLM-Planner; condition Abstract tasks / L2; column Rejection ratio |
| Llama3-8B / LLM-Planner | Rejection ratio | 0.00ratio | Abstract tasks / L3#A4.T15; row Llama3-8B / LLM-Planner; condition Abstract tasks / L3; column Rejection ratio |
| Llama3-8B / LLM-Planner | Rejection ratio | 0.00ratio | Abstract tasks / L4#A4.T15; row Llama3-8B / LLM-Planner; condition Abstract tasks / L4; column Rejection ratio |
| Llama3-8B / LLM-Planner | Risk ratio | 0.11ratio | Abstract tasks / L1#A4.T15; row Llama3-8B / LLM-Planner; condition Abstract tasks / L1; column Risk ratio |
| Llama3-8B / LLM-Planner | Risk ratio | 0.14ratio | Abstract tasks / L2#A4.T15; row Llama3-8B / LLM-Planner; condition Abstract tasks / L2; column Risk ratio |
| Llama3-8B / LLM-Planner | Risk ratio | 0.14ratio | Abstract tasks / L3#A4.T15; row Llama3-8B / LLM-Planner; condition Abstract tasks / L3; column Risk ratio |
| Llama3-8B / LLM-Planner | Risk ratio | 0.20ratio | Abstract tasks / L4#A4.T15; row Llama3-8B / LLM-Planner; condition Abstract tasks / L4; column Risk ratio |
| Llama3-8B / Co-LLM-Agent | Rejection ratio | 0.00ratio | Abstract tasks / L1#A4.T15; row Llama3-8B / Co-LLM-Agent; condition Abstract tasks / L1; column Rejection ratio |
| Llama3-8B / Co-LLM-Agent | Rejection ratio | 0.00ratio | Abstract tasks / L2#A4.T15; row Llama3-8B / Co-LLM-Agent; condition Abstract tasks / L2; column Rejection ratio |
| Llama3-8B / Co-LLM-Agent | Rejection ratio | 0.00ratio | Abstract tasks / L3#A4.T15; row Llama3-8B / Co-LLM-Agent; condition Abstract tasks / L3; column Rejection ratio |
| Llama3-8B / Co-LLM-Agent | Rejection ratio | 0.00ratio | Abstract tasks / L4#A4.T15; row Llama3-8B / Co-LLM-Agent; condition Abstract tasks / L4; column Rejection ratio |
| Llama3-8B / Co-LLM-Agent | Risk ratio | 0.02ratio | Abstract tasks / L1#A4.T15; row Llama3-8B / Co-LLM-Agent; condition Abstract tasks / L1; column Risk ratio |
| Llama3-8B / Co-LLM-Agent | Risk ratio | 0.00ratio | Abstract tasks / L2#A4.T15; row Llama3-8B / Co-LLM-Agent; condition Abstract tasks / L2; column Risk ratio |
| Llama3-8B / Co-LLM-Agent | Risk ratio | 0.00ratio | Abstract tasks / L3#A4.T15; row Llama3-8B / Co-LLM-Agent; condition Abstract tasks / L3; column Risk ratio |
| Llama3-8B / Co-LLM-Agent | Risk ratio | 0.06ratio | Abstract tasks / L4#A4.T15; row Llama3-8B / Co-LLM-Agent; condition Abstract tasks / L4; column Risk ratio |
| Llama3-8B / MLDT | Rejection ratio | 0.08ratio | Abstract tasks / L1#A4.T15; row Llama3-8B / MLDT; condition Abstract tasks / L1; column Rejection ratio |
| Llama3-8B / MLDT | Rejection ratio | 0.17ratio | Abstract tasks / L2#A4.T15; row Llama3-8B / MLDT; condition Abstract tasks / L2; column Rejection ratio |
| Llama3-8B / MLDT | Rejection ratio | 0.55ratio | Abstract tasks / L3#A4.T15; row Llama3-8B / MLDT; condition Abstract tasks / L3; column Rejection ratio |
| Llama3-8B / MLDT | Rejection ratio | 0.63ratio | Abstract tasks / L4#A4.T15; row Llama3-8B / MLDT; condition Abstract tasks / L4; column Rejection ratio |
| Llama3-8B / MLDT | Risk ratio | 0.44ratio | Abstract tasks / L1#A4.T15; row Llama3-8B / MLDT; condition Abstract tasks / L1; column Risk ratio |
| Llama3-8B / MLDT | Risk ratio | 0.21ratio | Abstract tasks / L2#A4.T15; row Llama3-8B / MLDT; condition Abstract tasks / L2; column Risk ratio |
| Llama3-8B / MLDT | Risk ratio | 0.18ratio | Abstract tasks / L3#A4.T15; row Llama3-8B / MLDT; condition Abstract tasks / L3; column Risk ratio |
| Llama3-8B / MLDT | Risk ratio | 0.31ratio | Abstract tasks / L4#A4.T15; row Llama3-8B / MLDT; condition Abstract tasks / L4; column Risk ratio |
| Llama3-8B / ProgPrompt | Rejection ratio | 0.03ratio | Abstract tasks / L1#A4.T15; row Llama3-8B / ProgPrompt; condition Abstract tasks / L1; column Rejection ratio |
| Llama3-8B / ProgPrompt | Rejection ratio | 0.00ratio | Abstract tasks / L2#A4.T15; row Llama3-8B / ProgPrompt; condition Abstract tasks / L2; column Rejection ratio |
| Llama3-8B / ProgPrompt | Rejection ratio | 0.06ratio | Abstract tasks / L3#A4.T15; row Llama3-8B / ProgPrompt; condition Abstract tasks / L3; column Rejection ratio |
| Llama3-8B / ProgPrompt | Rejection ratio | 0.26ratio | Abstract tasks / L4#A4.T15; row Llama3-8B / ProgPrompt; condition Abstract tasks / L4; column Rejection ratio |
| Llama3-8B / ProgPrompt | Risk ratio | 0.46ratio | Abstract tasks / L1#A4.T15; row Llama3-8B / ProgPrompt; condition Abstract tasks / L1; column Risk ratio |
| Llama3-8B / ProgPrompt | Risk ratio | 0.47ratio | Abstract tasks / L2#A4.T15; row Llama3-8B / ProgPrompt; condition Abstract tasks / L2; column Risk ratio |
| Llama3-8B / ProgPrompt | Risk ratio | 0.21ratio | Abstract tasks / L3#A4.T15; row Llama3-8B / ProgPrompt; condition Abstract tasks / L3; column Risk ratio |
| Llama3-8B / ProgPrompt | Risk ratio | 0.15ratio | Abstract tasks / L4#A4.T15; row Llama3-8B / ProgPrompt; condition Abstract tasks / L4; column Risk ratio |
| Llama3-8B / ReAct | Rejection ratio | 0.00ratio | Abstract tasks / L1#A4.T15; row Llama3-8B / ReAct; condition Abstract tasks / L1; column Rejection ratio |
| Llama3-8B / ReAct | Rejection ratio | 0.00ratio | Abstract tasks / L2#A4.T15; row Llama3-8B / ReAct; condition Abstract tasks / L2; column Rejection ratio |
| Llama3-8B / ReAct | Rejection ratio | 0.00ratio | Abstract tasks / L3#A4.T15; row Llama3-8B / ReAct; condition Abstract tasks / L3; column Rejection ratio |
| Llama3-8B / ReAct | Rejection ratio | 0.00ratio | Abstract tasks / L4#A4.T15; row Llama3-8B / ReAct; condition Abstract tasks / L4; column Rejection ratio |
| Llama3-8B / ReAct | Risk ratio | 0.07ratio | Abstract tasks / L1#A4.T15; row Llama3-8B / ReAct; condition Abstract tasks / L1; column Risk ratio |
| Llama3-8B / ReAct | Risk ratio | 0.05ratio | Abstract tasks / L2#A4.T15; row Llama3-8B / ReAct; condition Abstract tasks / L2; column Risk ratio |
| Llama3-8B / ReAct | Risk ratio | 0.06ratio | Abstract tasks / L3#A4.T15; row Llama3-8B / ReAct; condition Abstract tasks / L3; column Risk ratio |
| Llama3-8B / ReAct | Risk ratio | 0.04ratio | Abstract tasks / L4#A4.T15; row Llama3-8B / ReAct; condition Abstract tasks / L4; column Risk ratio |
| Llama3-8B / PCA-EVAL | Rejection ratio | 0.00ratio | Abstract tasks / L1#A4.T15; row Llama3-8B / PCA-EVAL; condition Abstract tasks / L1; column Rejection ratio |
| Llama3-8B / PCA-EVAL | Rejection ratio | 0.00ratio | Abstract tasks / L2#A4.T15; row Llama3-8B / PCA-EVAL; condition Abstract tasks / L2; column Rejection ratio |
| Llama3-8B / PCA-EVAL | Rejection ratio | 0.00ratio | Abstract tasks / L3#A4.T15; row Llama3-8B / PCA-EVAL; condition Abstract tasks / L3; column Rejection ratio |
| Llama3-8B / PCA-EVAL | Rejection ratio | 0.00ratio | Abstract tasks / L4#A4.T15; row Llama3-8B / PCA-EVAL; condition Abstract tasks / L4; column Rejection ratio |
| Llama3-8B / PCA-EVAL | Risk ratio | 0.06ratio | Abstract tasks / L1#A4.T15; row Llama3-8B / PCA-EVAL; condition Abstract tasks / L1; column Risk ratio |
| Llama3-8B / PCA-EVAL | Risk ratio | 0.02ratio | Abstract tasks / L2#A4.T15; row Llama3-8B / PCA-EVAL; condition Abstract tasks / L2; column Risk ratio |
| Llama3-8B / PCA-EVAL | Risk ratio | 0.03ratio | Abstract tasks / L3#A4.T15; row Llama3-8B / PCA-EVAL; condition Abstract tasks / L3; column Risk ratio |
| Llama3-8B / PCA-EVAL | Risk ratio | 0.04ratio | Abstract tasks / L4#A4.T15; row Llama3-8B / PCA-EVAL; condition Abstract tasks / L4; column Risk ratio |
| Qwen2-7B / Lota-Bench | Rejection ratio | 0.00ratio | Abstract tasks / L1#A4.T16; row Qwen2-7B / Lota-Bench; condition Abstract tasks / L1; column Rejection ratio |
| Qwen2-7B / Lota-Bench | Rejection ratio | 0.00ratio | Abstract tasks / L2#A4.T16; row Qwen2-7B / Lota-Bench; condition Abstract tasks / L2; column Rejection ratio |
| Qwen2-7B / Lota-Bench | Rejection ratio | 0.03ratio | Abstract tasks / L3#A4.T16; row Qwen2-7B / Lota-Bench; condition Abstract tasks / L3; column Rejection ratio |
| Qwen2-7B / Lota-Bench | Rejection ratio | 0.16ratio | Abstract tasks / L4#A4.T16; row Qwen2-7B / Lota-Bench; condition Abstract tasks / L4; column Rejection ratio |
| Qwen2-7B / Lota-Bench | Risk ratio | 0.29ratio | Abstract tasks / L1#A4.T16; row Qwen2-7B / Lota-Bench; condition Abstract tasks / L1; column Risk ratio |
| Qwen2-7B / Lota-Bench | Risk ratio | 0.32ratio | Abstract tasks / L2#A4.T16; row Qwen2-7B / Lota-Bench; condition Abstract tasks / L2; column Risk ratio |
| Qwen2-7B / Lota-Bench | Risk ratio | 0.19ratio | Abstract tasks / L3#A4.T16; row Qwen2-7B / Lota-Bench; condition Abstract tasks / L3; column Risk ratio |
| Qwen2-7B / Lota-Bench | Risk ratio | 0.21ratio | Abstract tasks / L4#A4.T16; row Qwen2-7B / Lota-Bench; condition Abstract tasks / L4; column Risk ratio |
| Qwen2-7B / LLM-Planner | Rejection ratio | 0.00ratio | Abstract tasks / L1#A4.T16; row Qwen2-7B / LLM-Planner; condition Abstract tasks / L1; column Rejection ratio |
| Qwen2-7B / LLM-Planner | Rejection ratio | 0.00ratio | Abstract tasks / L2#A4.T16; row Qwen2-7B / LLM-Planner; condition Abstract tasks / L2; column Rejection ratio |
| Qwen2-7B / LLM-Planner | Rejection ratio | 0.00ratio | Abstract tasks / L3#A4.T16; row Qwen2-7B / LLM-Planner; condition Abstract tasks / L3; column Rejection ratio |
| Qwen2-7B / LLM-Planner | Rejection ratio | 0.00ratio | Abstract tasks / L4#A4.T16; row Qwen2-7B / LLM-Planner; condition Abstract tasks / L4; column Rejection ratio |
| Qwen2-7B / LLM-Planner | Risk ratio | 0.18ratio | Abstract tasks / L1#A4.T16; row Qwen2-7B / LLM-Planner; condition Abstract tasks / L1; column Risk ratio |
| Qwen2-7B / LLM-Planner | Risk ratio | 0.22ratio | Abstract tasks / L2#A4.T16; row Qwen2-7B / LLM-Planner; condition Abstract tasks / L2; column Risk ratio |
| Qwen2-7B / LLM-Planner | Risk ratio | 0.03ratio | Abstract tasks / L3#A4.T16; row Qwen2-7B / LLM-Planner; condition Abstract tasks / L3; column Risk ratio |
| Qwen2-7B / LLM-Planner | Risk ratio | 0.13ratio | Abstract tasks / L4#A4.T16; row Qwen2-7B / LLM-Planner; condition Abstract tasks / L4; column Risk ratio |
| Qwen2-7B / Co-LLM-Agent | Rejection ratio | 0.00ratio | Abstract tasks / L1#A4.T16; row Qwen2-7B / Co-LLM-Agent; condition Abstract tasks / L1; column Rejection ratio |
| Qwen2-7B / Co-LLM-Agent | Rejection ratio | 0.00ratio | Abstract tasks / L2#A4.T16; row Qwen2-7B / Co-LLM-Agent; condition Abstract tasks / L2; column Rejection ratio |
| Qwen2-7B / Co-LLM-Agent | Rejection ratio | 0.00ratio | Abstract tasks / L3#A4.T16; row Qwen2-7B / Co-LLM-Agent; condition Abstract tasks / L3; column Rejection ratio |
| Qwen2-7B / Co-LLM-Agent | Rejection ratio | 0.00ratio | Abstract tasks / L4#A4.T16; row Qwen2-7B / Co-LLM-Agent; condition Abstract tasks / L4; column Rejection ratio |
| Qwen2-7B / Co-LLM-Agent | Risk ratio | 0.06ratio | Abstract tasks / L1#A4.T16; row Qwen2-7B / Co-LLM-Agent; condition Abstract tasks / L1; column Risk ratio |
| Qwen2-7B / Co-LLM-Agent | Risk ratio | 0.06ratio | Abstract tasks / L2#A4.T16; row Qwen2-7B / Co-LLM-Agent; condition Abstract tasks / L2; column Risk ratio |
| Qwen2-7B / Co-LLM-Agent | Risk ratio | 0.04ratio | Abstract tasks / L3#A4.T16; row Qwen2-7B / Co-LLM-Agent; condition Abstract tasks / L3; column Risk ratio |
| Qwen2-7B / Co-LLM-Agent | Risk ratio | 0.04ratio | Abstract tasks / L4#A4.T16; row Qwen2-7B / Co-LLM-Agent; condition Abstract tasks / L4; column Risk ratio |
| Qwen2-7B / MLDT | Rejection ratio | 0.03ratio | Abstract tasks / L1#A4.T16; row Qwen2-7B / MLDT; condition Abstract tasks / L1; column Rejection ratio |
| Qwen2-7B / MLDT | Rejection ratio | 0.00ratio | Abstract tasks / L2#A4.T16; row Qwen2-7B / MLDT; condition Abstract tasks / L2; column Rejection ratio |
| Qwen2-7B / MLDT | Rejection ratio | 0.02ratio | Abstract tasks / L3#A4.T16; row Qwen2-7B / MLDT; condition Abstract tasks / L3; column Rejection ratio |
| Qwen2-7B / MLDT | Rejection ratio | 0.00ratio | Abstract tasks / L4#A4.T16; row Qwen2-7B / MLDT; condition Abstract tasks / L4; column Rejection ratio |
| Qwen2-7B / MLDT | Risk ratio | 0.25ratio | Abstract tasks / L1#A4.T16; row Qwen2-7B / MLDT; condition Abstract tasks / L1; column Risk ratio |
| Qwen2-7B / MLDT | Risk ratio | 0.55ratio | Abstract tasks / L2#A4.T16; row Qwen2-7B / MLDT; condition Abstract tasks / L2; column Risk ratio |
| Qwen2-7B / MLDT | Risk ratio | 0.20ratio | Abstract tasks / L3#A4.T16; row Qwen2-7B / MLDT; condition Abstract tasks / L3; column Risk ratio |
| Qwen2-7B / MLDT | Risk ratio | 0.10ratio | Abstract tasks / L4#A4.T16; row Qwen2-7B / MLDT; condition Abstract tasks / L4; column Risk ratio |
| Qwen2-7B / ProgPrompt | Rejection ratio | 0.07ratio | Abstract tasks / L1#A4.T16; row Qwen2-7B / ProgPrompt; condition Abstract tasks / L1; column Rejection ratio |
| Qwen2-7B / ProgPrompt | Rejection ratio | 0.08ratio | Abstract tasks / L2#A4.T16; row Qwen2-7B / ProgPrompt; condition Abstract tasks / L2; column Rejection ratio |
| Qwen2-7B / ProgPrompt | Rejection ratio | 0.09ratio | Abstract tasks / L3#A4.T16; row Qwen2-7B / ProgPrompt; condition Abstract tasks / L3; column Rejection ratio |
| Qwen2-7B / ProgPrompt | Rejection ratio | 0.15ratio | Abstract tasks / L4#A4.T16; row Qwen2-7B / ProgPrompt; condition Abstract tasks / L4; column Rejection ratio |
| Qwen2-7B / ProgPrompt | Risk ratio | 0.42ratio | Abstract tasks / L1#A4.T16; row Qwen2-7B / ProgPrompt; condition Abstract tasks / L1; column Risk ratio |
| Qwen2-7B / ProgPrompt | Risk ratio | 0.26ratio | Abstract tasks / L2#A4.T16; row Qwen2-7B / ProgPrompt; condition Abstract tasks / L2; column Risk ratio |
| Qwen2-7B / ProgPrompt | Risk ratio | 0.20ratio | Abstract tasks / L3#A4.T16; row Qwen2-7B / ProgPrompt; condition Abstract tasks / L3; column Risk ratio |
| Qwen2-7B / ProgPrompt | Risk ratio | 0.25ratio | Abstract tasks / L4#A4.T16; row Qwen2-7B / ProgPrompt; condition Abstract tasks / L4; column Risk ratio |
| Qwen2-7B / ReAct | Rejection ratio | 0.00ratio | Abstract tasks / L1#A4.T16; row Qwen2-7B / ReAct; condition Abstract tasks / L1; column Rejection ratio |
| Qwen2-7B / ReAct | Rejection ratio | 0.00ratio | Abstract tasks / L2#A4.T16; row Qwen2-7B / ReAct; condition Abstract tasks / L2; column Rejection ratio |
| Qwen2-7B / ReAct | Rejection ratio | 0.00ratio | Abstract tasks / L3#A4.T16; row Qwen2-7B / ReAct; condition Abstract tasks / L3; column Rejection ratio |
| Qwen2-7B / ReAct | Rejection ratio | 0.00ratio | Abstract tasks / L4#A4.T16; row Qwen2-7B / ReAct; condition Abstract tasks / L4; column Rejection ratio |
| Qwen2-7B / ReAct | Risk ratio | 0.02ratio | Abstract tasks / L1#A4.T16; row Qwen2-7B / ReAct; condition Abstract tasks / L1; column Risk ratio |
| Qwen2-7B / ReAct | Risk ratio | 0.05ratio | Abstract tasks / L2#A4.T16; row Qwen2-7B / ReAct; condition Abstract tasks / L2; column Risk ratio |
| Qwen2-7B / ReAct | Risk ratio | 0.08ratio | Abstract tasks / L3#A4.T16; row Qwen2-7B / ReAct; condition Abstract tasks / L3; column Risk ratio |
| Qwen2-7B / ReAct | Risk ratio | 0.04ratio | Abstract tasks / L4#A4.T16; row Qwen2-7B / ReAct; condition Abstract tasks / L4; column Risk ratio |
| Qwen2-7B / PCA-EVAL | Rejection ratio | 0.00ratio | Abstract tasks / L1#A4.T16; row Qwen2-7B / PCA-EVAL; condition Abstract tasks / L1; column Rejection ratio |
| Qwen2-7B / PCA-EVAL | Rejection ratio | 0.00ratio | Abstract tasks / L2#A4.T16; row Qwen2-7B / PCA-EVAL; condition Abstract tasks / L2; column Rejection ratio |
| Qwen2-7B / PCA-EVAL | Rejection ratio | 0.00ratio | Abstract tasks / L3#A4.T16; row Qwen2-7B / PCA-EVAL; condition Abstract tasks / L3; column Rejection ratio |
| Qwen2-7B / PCA-EVAL | Rejection ratio | 0.00ratio | Abstract tasks / L4#A4.T16; row Qwen2-7B / PCA-EVAL; condition Abstract tasks / L4; column Rejection ratio |
| Qwen2-7B / PCA-EVAL | Risk ratio | 0.12ratio | Abstract tasks / L1#A4.T16; row Qwen2-7B / PCA-EVAL; condition Abstract tasks / L1; column Risk ratio |
| Qwen2-7B / PCA-EVAL | Risk ratio | 0.05ratio | Abstract tasks / L2#A4.T16; row Qwen2-7B / PCA-EVAL; condition Abstract tasks / L2; column Risk ratio |
| Qwen2-7B / PCA-EVAL | Risk ratio | 0.04ratio | Abstract tasks / L3#A4.T16; row Qwen2-7B / PCA-EVAL; condition Abstract tasks / L3; column Risk ratio |
| Qwen2-7B / PCA-EVAL | Risk ratio | 0.12ratio | Abstract tasks / L4#A4.T16; row Qwen2-7B / PCA-EVAL; condition Abstract tasks / L4; column Risk ratio |
| DeepSeek-V2.5 / Lota-Bench | Rejection ratio | 0.00ratio | Abstract tasks / L1#A4.T17; row DeepSeek-V2.5 / Lota-Bench; condition Abstract tasks / L1; column Rejection ratio |
| DeepSeek-V2.5 / Lota-Bench | Rejection ratio | 0.00ratio | Abstract tasks / L2#A4.T17; row DeepSeek-V2.5 / Lota-Bench; condition Abstract tasks / L2; column Rejection ratio |
| DeepSeek-V2.5 / Lota-Bench | Rejection ratio | 0.00ratio | Abstract tasks / L3#A4.T17; row DeepSeek-V2.5 / Lota-Bench; condition Abstract tasks / L3; column Rejection ratio |
| DeepSeek-V2.5 / Lota-Bench | Rejection ratio | 0.02ratio | Abstract tasks / L4#A4.T17; row DeepSeek-V2.5 / Lota-Bench; condition Abstract tasks / L4; column Rejection ratio |
| DeepSeek-V2.5 / Lota-Bench | Risk ratio | 0.82ratio | Abstract tasks / L1#A4.T17; row DeepSeek-V2.5 / Lota-Bench; condition Abstract tasks / L1; column Risk ratio |
| DeepSeek-V2.5 / Lota-Bench | Risk ratio | 0.58ratio | Abstract tasks / L2#A4.T17; row DeepSeek-V2.5 / Lota-Bench; condition Abstract tasks / L2; column Risk ratio |
| DeepSeek-V2.5 / Lota-Bench | Risk ratio | 0.56ratio | Abstract tasks / L3#A4.T17; row DeepSeek-V2.5 / Lota-Bench; condition Abstract tasks / L3; column Risk ratio |
| DeepSeek-V2.5 / Lota-Bench | Risk ratio | 0.66ratio | Abstract tasks / L4#A4.T17; row DeepSeek-V2.5 / Lota-Bench; condition Abstract tasks / L4; column Risk ratio |
| DeepSeek-V2.5 / LLM-Planner | Rejection ratio | 0.00ratio | Abstract tasks / L1#A4.T17; row DeepSeek-V2.5 / LLM-Planner; condition Abstract tasks / L1; column Rejection ratio |
| DeepSeek-V2.5 / LLM-Planner | Rejection ratio | 0.00ratio | Abstract tasks / L2#A4.T17; row DeepSeek-V2.5 / LLM-Planner; condition Abstract tasks / L2; column Rejection ratio |
| DeepSeek-V2.5 / LLM-Planner | Rejection ratio | 0.00ratio | Abstract tasks / L3#A4.T17; row DeepSeek-V2.5 / LLM-Planner; condition Abstract tasks / L3; column Rejection ratio |
| DeepSeek-V2.5 / LLM-Planner | Rejection ratio | 0.00ratio | Abstract tasks / L4#A4.T17; row DeepSeek-V2.5 / LLM-Planner; condition Abstract tasks / L4; column Rejection ratio |
| DeepSeek-V2.5 / LLM-Planner | Risk ratio | 0.48ratio | Abstract tasks / L1#A4.T17; row DeepSeek-V2.5 / LLM-Planner; condition Abstract tasks / L1; column Risk ratio |
| DeepSeek-V2.5 / LLM-Planner | Risk ratio | 0.35ratio | Abstract tasks / L2#A4.T17; row DeepSeek-V2.5 / LLM-Planner; condition Abstract tasks / L2; column Risk ratio |
| DeepSeek-V2.5 / LLM-Planner | Risk ratio | 0.32ratio | Abstract tasks / L3#A4.T17; row DeepSeek-V2.5 / LLM-Planner; condition Abstract tasks / L3; column Risk ratio |
| DeepSeek-V2.5 / LLM-Planner | Risk ratio | 0.19ratio | Abstract tasks / L4#A4.T17; row DeepSeek-V2.5 / LLM-Planner; condition Abstract tasks / L4; column Risk ratio |
| DeepSeek-V2.5 / Co-LLM-Agent | Rejection ratio | 0.00ratio | Abstract tasks / L1#A4.T17; row DeepSeek-V2.5 / Co-LLM-Agent; condition Abstract tasks / L1; column Rejection ratio |
| DeepSeek-V2.5 / Co-LLM-Agent | Rejection ratio | 0.00ratio | Abstract tasks / L2#A4.T17; row DeepSeek-V2.5 / Co-LLM-Agent; condition Abstract tasks / L2; column Rejection ratio |
| DeepSeek-V2.5 / Co-LLM-Agent | Rejection ratio | 0.00ratio | Abstract tasks / L3#A4.T17; row DeepSeek-V2.5 / Co-LLM-Agent; condition Abstract tasks / L3; column Rejection ratio |
| DeepSeek-V2.5 / Co-LLM-Agent | Rejection ratio | 0.00ratio | Abstract tasks / L4#A4.T17; row DeepSeek-V2.5 / Co-LLM-Agent; condition Abstract tasks / L4; column Rejection ratio |
| DeepSeek-V2.5 / Co-LLM-Agent | Risk ratio | 0.22ratio | Abstract tasks / L1#A4.T17; row DeepSeek-V2.5 / Co-LLM-Agent; condition Abstract tasks / L1; column Risk ratio |
| DeepSeek-V2.5 / Co-LLM-Agent | Risk ratio | 0.08ratio | Abstract tasks / L2#A4.T17; row DeepSeek-V2.5 / Co-LLM-Agent; condition Abstract tasks / L2; column Risk ratio |
| DeepSeek-V2.5 / Co-LLM-Agent | Risk ratio | 0.04ratio | Abstract tasks / L3#A4.T17; row DeepSeek-V2.5 / Co-LLM-Agent; condition Abstract tasks / L3; column Risk ratio |
| DeepSeek-V2.5 / Co-LLM-Agent | Risk ratio | 0.04ratio | Abstract tasks / L4#A4.T17; row DeepSeek-V2.5 / Co-LLM-Agent; condition Abstract tasks / L4; column Risk ratio |
| DeepSeek-V2.5 / MLDT | Rejection ratio | 0.03ratio | Abstract tasks / L1#A4.T17; row DeepSeek-V2.5 / MLDT; condition Abstract tasks / L1; column Rejection ratio |
| DeepSeek-V2.5 / MLDT | Rejection ratio | 0.00ratio | Abstract tasks / L2#A4.T17; row DeepSeek-V2.5 / MLDT; condition Abstract tasks / L2; column Rejection ratio |
| DeepSeek-V2.5 / MLDT | Rejection ratio | 0.02ratio | Abstract tasks / L3#A4.T17; row DeepSeek-V2.5 / MLDT; condition Abstract tasks / L3; column Rejection ratio |
| DeepSeek-V2.5 / MLDT | Rejection ratio | 0.00ratio | Abstract tasks / L4#A4.T17; row DeepSeek-V2.5 / MLDT; condition Abstract tasks / L4; column Rejection ratio |
| DeepSeek-V2.5 / MLDT | Risk ratio | 0.70ratio | Abstract tasks / L1#A4.T17; row DeepSeek-V2.5 / MLDT; condition Abstract tasks / L1; column Risk ratio |
| DeepSeek-V2.5 / MLDT | Risk ratio | 0.50ratio | Abstract tasks / L2#A4.T17; row DeepSeek-V2.5 / MLDT; condition Abstract tasks / L2; column Risk ratio |
| DeepSeek-V2.5 / MLDT | Risk ratio | 0.40ratio | Abstract tasks / L3#A4.T17; row DeepSeek-V2.5 / MLDT; condition Abstract tasks / L3; column Risk ratio |
| DeepSeek-V2.5 / MLDT | Risk ratio | 0.45ratio | Abstract tasks / L4#A4.T17; row DeepSeek-V2.5 / MLDT; condition Abstract tasks / L4; column Risk ratio |
| DeepSeek-V2.5 / ProgPrompt | Rejection ratio | 0.03ratio | Abstract tasks / L1#A4.T17; row DeepSeek-V2.5 / ProgPrompt; condition Abstract tasks / L1; column Rejection ratio |
| DeepSeek-V2.5 / ProgPrompt | Rejection ratio | 0.07ratio | Abstract tasks / L2#A4.T17; row DeepSeek-V2.5 / ProgPrompt; condition Abstract tasks / L2; column Rejection ratio |
| DeepSeek-V2.5 / ProgPrompt | Rejection ratio | 0.13ratio | Abstract tasks / L3#A4.T17; row DeepSeek-V2.5 / ProgPrompt; condition Abstract tasks / L3; column Rejection ratio |
| DeepSeek-V2.5 / ProgPrompt | Rejection ratio | 0.13ratio | Abstract tasks / L4#A4.T17; row DeepSeek-V2.5 / ProgPrompt; condition Abstract tasks / L4; column Rejection ratio |
| DeepSeek-V2.5 / ProgPrompt | Risk ratio | 0.75ratio | Abstract tasks / L1#A4.T17; row DeepSeek-V2.5 / ProgPrompt; condition Abstract tasks / L1; column Risk ratio |
| DeepSeek-V2.5 / ProgPrompt | Risk ratio | 0.65ratio | Abstract tasks / L2#A4.T17; row DeepSeek-V2.5 / ProgPrompt; condition Abstract tasks / L2; column Risk ratio |
| DeepSeek-V2.5 / ProgPrompt | Risk ratio | 0.55ratio | Abstract tasks / L3#A4.T17; row DeepSeek-V2.5 / ProgPrompt; condition Abstract tasks / L3; column Risk ratio |
| DeepSeek-V2.5 / ProgPrompt | Risk ratio | 0.45ratio | Abstract tasks / L4#A4.T17; row DeepSeek-V2.5 / ProgPrompt; condition Abstract tasks / L4; column Risk ratio |
| DeepSeek-V2.5 / ReAct | Rejection ratio | 0.00ratio | Abstract tasks / L1#A4.T17; row DeepSeek-V2.5 / ReAct; condition Abstract tasks / L1; column Rejection ratio |
| DeepSeek-V2.5 / ReAct | Rejection ratio | 0.00ratio | Abstract tasks / L2#A4.T17; row DeepSeek-V2.5 / ReAct; condition Abstract tasks / L2; column Rejection ratio |
| DeepSeek-V2.5 / ReAct | Rejection ratio | 0.00ratio | Abstract tasks / L3#A4.T17; row DeepSeek-V2.5 / ReAct; condition Abstract tasks / L3; column Rejection ratio |
| DeepSeek-V2.5 / ReAct | Rejection ratio | 0.00ratio | Abstract tasks / L4#A4.T17; row DeepSeek-V2.5 / ReAct; condition Abstract tasks / L4; column Rejection ratio |
| DeepSeek-V2.5 / ReAct | Risk ratio | 0.24ratio | Abstract tasks / L1#A4.T17; row DeepSeek-V2.5 / ReAct; condition Abstract tasks / L1; column Risk ratio |
| DeepSeek-V2.5 / ReAct | Risk ratio | 0.18ratio | Abstract tasks / L2#A4.T17; row DeepSeek-V2.5 / ReAct; condition Abstract tasks / L2; column Risk ratio |
| DeepSeek-V2.5 / ReAct | Risk ratio | 0.22ratio | Abstract tasks / L3#A4.T17; row DeepSeek-V2.5 / ReAct; condition Abstract tasks / L3; column Risk ratio |
| DeepSeek-V2.5 / ReAct | Risk ratio | 0.17ratio | Abstract tasks / L4#A4.T17; row DeepSeek-V2.5 / ReAct; condition Abstract tasks / L4; column Risk ratio |
| DeepSeek-V2.5 / PCA-EVAL | Rejection ratio | 0.00ratio | Abstract tasks / L1#A4.T17; row DeepSeek-V2.5 / PCA-EVAL; condition Abstract tasks / L1; column Rejection ratio |
| DeepSeek-V2.5 / PCA-EVAL | Rejection ratio | 0.00ratio | Abstract tasks / L2#A4.T17; row DeepSeek-V2.5 / PCA-EVAL; condition Abstract tasks / L2; column Rejection ratio |
| DeepSeek-V2.5 / PCA-EVAL | Rejection ratio | 0.00ratio | Abstract tasks / L3#A4.T17; row DeepSeek-V2.5 / PCA-EVAL; condition Abstract tasks / L3; column Rejection ratio |
| DeepSeek-V2.5 / PCA-EVAL | Rejection ratio | 0.00ratio | Abstract tasks / L4#A4.T17; row DeepSeek-V2.5 / PCA-EVAL; condition Abstract tasks / L4; column Rejection ratio |
| DeepSeek-V2.5 / PCA-EVAL | Risk ratio | 0.28ratio | Abstract tasks / L1#A4.T17; row DeepSeek-V2.5 / PCA-EVAL; condition Abstract tasks / L1; column Risk ratio |
| DeepSeek-V2.5 / PCA-EVAL | Risk ratio | 0.11ratio | Abstract tasks / L2#A4.T17; row DeepSeek-V2.5 / PCA-EVAL; condition Abstract tasks / L2; column Risk ratio |
| DeepSeek-V2.5 / PCA-EVAL | Risk ratio | 0.22ratio | Abstract tasks / L3#A4.T17; row DeepSeek-V2.5 / PCA-EVAL; condition Abstract tasks / L3; column Risk ratio |
| DeepSeek-V2.5 / PCA-EVAL | Risk ratio | 0.17ratio | Abstract tasks / L4#A4.T17; row DeepSeek-V2.5 / PCA-EVAL; condition Abstract tasks / L4; column Risk ratio |
阅读范围与复核
Selected original HTML sections and table context; not a complete paper/appendix audit or independent reproduction.
模型辅助整理，独立证据复核待完成。
核对原始来源 ↗