NUWA RESEARCH RECORD
SafeAgentBench: A Benchmark for Safe Task Planning of Embodied LLM Agents
Original publication ↗ · BibTeX · RISThe revised paper reports weak hazard awareness across planning approaches, despite differences in task completion. Changing the underlying LLM alone does not reliably solve the problem.
Research question
Can embodied planners recognize hazardous instructions while completing safe tasks?
Design & assumptions
Executable household simulations distinguish detailed, abstract, and long-horizon tasks; execution and semantic judgments provide complementary measurements.
Findings
The revised paper reports weak hazard awareness across planning approaches, despite differences in task completion. Changing the underlying LLM alone does not reliably solve the problem.
Limits & counterevidence
The low-level controller maps high-level actions to simulator APIs; this is not validation of a physical robot or learned motor controller. Hazardous examples are not reproduced here.
Source evidence
| System | Measure | Reported value | Conditions & location |
|---|---|---|---|
| GPT-4 / Lota-Bench | Rejection ratio | 0.00ratio | Detailed hazardous tasks#S4.T2; row GPT-4 / Lota-Bench; condition Detailed hazardous tasks; column Rejection ratio |
| GPT-4 / Lota-Bench | Risk ratio (goal evaluator) | 0.60ratio | Detailed hazardous tasks#S4.T2; row GPT-4 / Lota-Bench; condition Detailed hazardous tasks; column Risk ratio (goal evaluator) |
| GPT-4 / Lota-Bench | Risk ratio (LLM evaluator) | 0.38ratio | Detailed hazardous tasks#S4.T2; row GPT-4 / Lota-Bench; condition Detailed hazardous tasks; column Risk ratio (LLM evaluator) |
| GPT-4 / Lota-Bench | Execution ratio | 0.89ratio | Detailed hazardous tasks#S4.T2; row GPT-4 / Lota-Bench; condition Detailed hazardous tasks; column Execution ratio |
| GPT-4 / Lota-Bench | Execution time | 20.78s | Detailed hazardous tasks#S4.T2; row GPT-4 / Lota-Bench; condition Detailed hazardous tasks; column Execution time |
| GPT-4 / Lota-Bench | Rejection ratio | 0.00ratio | Abstract hazardous tasks#S4.T2; row GPT-4 / Lota-Bench; condition Abstract hazardous tasks; column Rejection ratio |
| GPT-4 / Lota-Bench | Risk ratio | 0.59ratio | Abstract hazardous tasks#S4.T2; row GPT-4 / Lota-Bench; condition Abstract hazardous tasks; column Risk ratio |
| GPT-4 / Lota-Bench | Completed safe ratio | 0.49ratio | Long-horizon tasks#S4.T2; row GPT-4 / Lota-Bench; condition Long-horizon tasks; column Completed safe ratio |
| GPT-4 / Lota-Bench | Completed unsafe ratio | 0.28ratio | Long-horizon tasks#S4.T2; row GPT-4 / Lota-Bench; condition Long-horizon tasks; column Completed unsafe ratio |
| GPT-4 / Lota-Bench | Incomplete ratio | 0.23ratio | Long-horizon tasks#S4.T2; row GPT-4 / Lota-Bench; condition Long-horizon tasks; column Incomplete ratio |
| GPT-4 / LLM-Planner | Rejection ratio | 0.00ratio | Detailed hazardous tasks#S4.T2; row GPT-4 / LLM-Planner; condition Detailed hazardous tasks; column Rejection ratio |
| GPT-4 / LLM-Planner | Risk ratio (goal evaluator) | 0.40ratio | Detailed hazardous tasks#S4.T2; row GPT-4 / LLM-Planner; condition Detailed hazardous tasks; column Risk ratio (goal evaluator) |
| GPT-4 / LLM-Planner | Risk ratio (LLM evaluator) | 0.46ratio | Detailed hazardous tasks#S4.T2; row GPT-4 / LLM-Planner; condition Detailed hazardous tasks; column Risk ratio (LLM evaluator) |
| GPT-4 / LLM-Planner | Execution ratio | 0.75ratio | Detailed hazardous tasks#S4.T2; row GPT-4 / LLM-Planner; condition Detailed hazardous tasks; column Execution ratio |
| GPT-4 / LLM-Planner | Execution time | 58.75s | Detailed hazardous tasks#S4.T2; row GPT-4 / LLM-Planner; condition Detailed hazardous tasks; column Execution time |
| GPT-4 / LLM-Planner | Rejection ratio | 0.31ratio | Abstract hazardous tasks#S4.T2; row GPT-4 / LLM-Planner; condition Abstract hazardous tasks; column Rejection ratio |
| GPT-4 / LLM-Planner | Risk ratio | 0.32ratio | Abstract hazardous tasks#S4.T2; row GPT-4 / LLM-Planner; condition Abstract hazardous tasks; column Risk ratio |
| GPT-4 / LLM-Planner | Completed safe ratio | 0.14ratio | Long-horizon tasks#S4.T2; row GPT-4 / LLM-Planner; condition Long-horizon tasks; column Completed safe ratio |
| GPT-4 / LLM-Planner | Completed unsafe ratio | 0.14ratio | Long-horizon tasks#S4.T2; row GPT-4 / LLM-Planner; condition Long-horizon tasks; column Completed unsafe ratio |
| GPT-4 / LLM-Planner | Incomplete ratio | 0.72ratio | Long-horizon tasks#S4.T2; row GPT-4 / LLM-Planner; condition Long-horizon tasks; column Incomplete ratio |
| GPT-4 / CoELA | Rejection ratio | 0.00ratio | Detailed hazardous tasks#S4.T2; row GPT-4 / CoELA; condition Detailed hazardous tasks; column Rejection ratio |
| GPT-4 / CoELA | Risk ratio (goal evaluator) | 0.16ratio | Detailed hazardous tasks#S4.T2; row GPT-4 / CoELA; condition Detailed hazardous tasks; column Risk ratio (goal evaluator) |
| GPT-4 / CoELA | Risk ratio (LLM evaluator) | 0.09ratio | Detailed hazardous tasks#S4.T2; row GPT-4 / CoELA; condition Detailed hazardous tasks; column Risk ratio (LLM evaluator) |
| GPT-4 / CoELA | Execution ratio | 0.33ratio | Detailed hazardous tasks#S4.T2; row GPT-4 / CoELA; condition Detailed hazardous tasks; column Execution ratio |
| GPT-4 / CoELA | Execution time | 74.12s | Detailed hazardous tasks#S4.T2; row GPT-4 / CoELA; condition Detailed hazardous tasks; column Execution time |
| GPT-4 / CoELA | Rejection ratio | 0.00ratio | Abstract hazardous tasks#S4.T2; row GPT-4 / CoELA; condition Abstract hazardous tasks; column Rejection ratio |
| GPT-4 / CoELA | Risk ratio | 0.21ratio | Abstract hazardous tasks#S4.T2; row GPT-4 / CoELA; condition Abstract hazardous tasks; column Risk ratio |
| GPT-4 / CoELA | Completed safe ratio | 0.02ratio | Long-horizon tasks#S4.T2; row GPT-4 / CoELA; condition Long-horizon tasks; column Completed safe ratio |
| GPT-4 / CoELA | Completed unsafe ratio | 0.02ratio | Long-horizon tasks#S4.T2; row GPT-4 / CoELA; condition Long-horizon tasks; column Completed unsafe ratio |
| GPT-4 / CoELA | Incomplete ratio | 0.96ratio | Long-horizon tasks#S4.T2; row GPT-4 / CoELA; condition Long-horizon tasks; column Incomplete ratio |
| GPT-4 / MLDT | Rejection ratio | 0.05ratio | Detailed hazardous tasks#S4.T2; row GPT-4 / MLDT; condition Detailed hazardous tasks; column Rejection ratio |
| GPT-4 / MLDT | Risk ratio (goal evaluator) | 0.54ratio | Detailed hazardous tasks#S4.T2; row GPT-4 / MLDT; condition Detailed hazardous tasks; column Risk ratio (goal evaluator) |
| GPT-4 / MLDT | Risk ratio (LLM evaluator) | 0.69ratio | Detailed hazardous tasks#S4.T2; row GPT-4 / MLDT; condition Detailed hazardous tasks; column Risk ratio (LLM evaluator) |
| GPT-4 / MLDT | Execution ratio | 0.73ratio | Detailed hazardous tasks#S4.T2; row GPT-4 / MLDT; condition Detailed hazardous tasks; column Execution ratio |
| GPT-4 / MLDT | Execution time | 31.92s | Detailed hazardous tasks#S4.T2; row GPT-4 / MLDT; condition Detailed hazardous tasks; column Execution time |
| GPT-4 / MLDT | Rejection ratio | 0.15ratio | Abstract hazardous tasks#S4.T2; row GPT-4 / MLDT; condition Abstract hazardous tasks; column Rejection ratio |
| GPT-4 / MLDT | Risk ratio | 0.40ratio | Abstract hazardous tasks#S4.T2; row GPT-4 / MLDT; condition Abstract hazardous tasks; column Risk ratio |
| GPT-4 / MLDT | Completed safe ratio | 0.49ratio | Long-horizon tasks#S4.T2; row GPT-4 / MLDT; condition Long-horizon tasks; column Completed safe ratio |
| GPT-4 / MLDT | Completed unsafe ratio | 0.28ratio | Long-horizon tasks#S4.T2; row GPT-4 / MLDT; condition Long-horizon tasks; column Completed unsafe ratio |
| GPT-4 / MLDT | Incomplete ratio | 0.23ratio | Long-horizon tasks#S4.T2; row GPT-4 / MLDT; condition Long-horizon tasks; column Incomplete ratio |
| GPT-4 / ProgPrompt | Rejection ratio | 0.07ratio | Detailed hazardous tasks#S4.T2; row GPT-4 / ProgPrompt; condition Detailed hazardous tasks; column Rejection ratio |
| GPT-4 / ProgPrompt | Risk ratio (goal evaluator) | 0.51ratio | Detailed hazardous tasks#S4.T2; row GPT-4 / ProgPrompt; condition Detailed hazardous tasks; column Risk ratio (goal evaluator) |
| GPT-4 / ProgPrompt | Risk ratio (LLM evaluator) | 0.68ratio | Detailed hazardous tasks#S4.T2; row GPT-4 / ProgPrompt; condition Detailed hazardous tasks; column Risk ratio (LLM evaluator) |
| GPT-4 / ProgPrompt | Execution ratio | 0.30ratio | Detailed hazardous tasks#S4.T2; row GPT-4 / ProgPrompt; condition Detailed hazardous tasks; column Execution ratio |
| GPT-4 / ProgPrompt | Execution time | 22.98s | Detailed hazardous tasks#S4.T2; row GPT-4 / ProgPrompt; condition Detailed hazardous tasks; column Execution time |
| GPT-4 / ProgPrompt | Rejection ratio | 0.20ratio | Abstract hazardous tasks#S4.T2; row GPT-4 / ProgPrompt; condition Abstract hazardous tasks; column Rejection ratio |
| GPT-4 / ProgPrompt | Risk ratio | 0.40ratio | Abstract hazardous tasks#S4.T2; row GPT-4 / ProgPrompt; condition Abstract hazardous tasks; column Risk ratio |
| GPT-4 / ProgPrompt | Completed safe ratio | 0.50ratio | Long-horizon tasks#S4.T2; row GPT-4 / ProgPrompt; condition Long-horizon tasks; column Completed safe ratio |
| GPT-4 / ProgPrompt | Completed unsafe ratio | 0.23ratio | Long-horizon tasks#S4.T2; row GPT-4 / ProgPrompt; condition Long-horizon tasks; column Completed unsafe ratio |
| GPT-4 / ProgPrompt | Incomplete ratio | 0.27ratio | Long-horizon tasks#S4.T2; row GPT-4 / ProgPrompt; condition Long-horizon tasks; column Incomplete ratio |
| GPT-4 / MAP | Rejection ratio | 0.00ratio | Detailed hazardous tasks#S4.T2; row GPT-4 / MAP; condition Detailed hazardous tasks; column Rejection ratio |
| GPT-4 / MAP | Risk ratio (goal evaluator) | 0.27ratio | Detailed hazardous tasks#S4.T2; row GPT-4 / MAP; condition Detailed hazardous tasks; column Risk ratio (goal evaluator) |
| GPT-4 / MAP | Risk ratio (LLM evaluator) | 0.31ratio | Detailed hazardous tasks#S4.T2; row GPT-4 / MAP; condition Detailed hazardous tasks; column Risk ratio (LLM evaluator) |
| GPT-4 / MAP | Execution ratio | 0.64ratio | Detailed hazardous tasks#S4.T2; row GPT-4 / MAP; condition Detailed hazardous tasks; column Execution ratio |
| GPT-4 / MAP | Execution time | 23.56s | Detailed hazardous tasks#S4.T2; row GPT-4 / MAP; condition Detailed hazardous tasks; column Execution time |
| GPT-4 / MAP | Rejection ratio | 0.00ratio | Abstract hazardous tasks#S4.T2; row GPT-4 / MAP; condition Abstract hazardous tasks; column Rejection ratio |
| GPT-4 / MAP | Risk ratio | 0.29ratio | Abstract hazardous tasks#S4.T2; row GPT-4 / MAP; condition Abstract hazardous tasks; column Risk ratio |
| GPT-4 / MAP | Completed safe ratio | 0.23ratio | Long-horizon tasks#S4.T2; row GPT-4 / MAP; condition Long-horizon tasks; column Completed safe ratio |
| GPT-4 / MAP | Completed unsafe ratio | 0.25ratio | Long-horizon tasks#S4.T2; row GPT-4 / MAP; condition Long-horizon tasks; column Completed unsafe ratio |
| GPT-4 / MAP | Incomplete ratio | 0.52ratio | Long-horizon tasks#S4.T2; row GPT-4 / MAP; condition Long-horizon tasks; column Incomplete ratio |
| GPT-4 / ReAct | Rejection ratio | 0.10ratio | Detailed hazardous tasks#S4.T2; row GPT-4 / ReAct; condition Detailed hazardous tasks; column Rejection ratio |
| GPT-4 / ReAct | Risk ratio (goal evaluator) | 0.42ratio | Detailed hazardous tasks#S4.T2; row GPT-4 / ReAct; condition Detailed hazardous tasks; column Risk ratio (goal evaluator) |
| GPT-4 / ReAct | Risk ratio (LLM evaluator) | 0.48ratio | Detailed hazardous tasks#S4.T2; row GPT-4 / ReAct; condition Detailed hazardous tasks; column Risk ratio (LLM evaluator) |
| GPT-4 / ReAct | Execution ratio | 0.74ratio | Detailed hazardous tasks#S4.T2; row GPT-4 / ReAct; condition Detailed hazardous tasks; column Execution ratio |
| GPT-4 / ReAct | Execution time | 26.95s | Detailed hazardous tasks#S4.T2; row GPT-4 / ReAct; condition Detailed hazardous tasks; column Execution time |
| GPT-4 / ReAct | Rejection ratio | 0.32ratio | Abstract hazardous tasks#S4.T2; row GPT-4 / ReAct; condition Abstract hazardous tasks; column Rejection ratio |
| GPT-4 / ReAct | Risk ratio | 0.55ratio | Abstract hazardous tasks#S4.T2; row GPT-4 / ReAct; condition Abstract hazardous tasks; column Risk ratio |
| GPT-4 / ReAct | Completed safe ratio | 0.04ratio | Long-horizon tasks#S4.T2; row GPT-4 / ReAct; condition Long-horizon tasks; column Completed safe ratio |
| GPT-4 / ReAct | Completed unsafe ratio | 0.08ratio | Long-horizon tasks#S4.T2; row GPT-4 / ReAct; condition Long-horizon tasks; column Completed unsafe ratio |
| GPT-4 / ReAct | Incomplete ratio | 0.88ratio | Long-horizon tasks#S4.T2; row GPT-4 / ReAct; condition Long-horizon tasks; column Incomplete ratio |
| GPT-4 / PCA-EVAL | Rejection ratio | 0.00ratio | Detailed hazardous tasks#S4.T2; row GPT-4 / PCA-EVAL; condition Detailed hazardous tasks; column Rejection ratio |
| GPT-4 / PCA-EVAL | Risk ratio (goal evaluator) | 0.36ratio | Detailed hazardous tasks#S4.T2; row GPT-4 / PCA-EVAL; condition Detailed hazardous tasks; column Risk ratio (goal evaluator) |
| GPT-4 / PCA-EVAL | Risk ratio (LLM evaluator) | 0.17ratio | Detailed hazardous tasks#S4.T2; row GPT-4 / PCA-EVAL; condition Detailed hazardous tasks; column Risk ratio (LLM evaluator) |
| GPT-4 / PCA-EVAL | Execution ratio | 0.85ratio | Detailed hazardous tasks#S4.T2; row GPT-4 / PCA-EVAL; condition Detailed hazardous tasks; column Execution ratio |
| GPT-4 / PCA-EVAL | Execution time | 97.30s | Detailed hazardous tasks#S4.T2; row GPT-4 / PCA-EVAL; condition Detailed hazardous tasks; column Execution time |
| GPT-4 / PCA-EVAL | Rejection ratio | 0.00ratio | Abstract hazardous tasks#S4.T2; row GPT-4 / PCA-EVAL; condition Abstract hazardous tasks; column Rejection ratio |
| GPT-4 / PCA-EVAL | Risk ratio | 0.17ratio | Abstract hazardous tasks#S4.T2; row GPT-4 / PCA-EVAL; condition Abstract hazardous tasks; column Risk ratio |
| GPT-4 / PCA-EVAL | Completed safe ratio | 0.06ratio | Long-horizon tasks#S4.T2; row GPT-4 / PCA-EVAL; condition Long-horizon tasks; column Completed safe ratio |
| GPT-4 / PCA-EVAL | Completed unsafe ratio | 0.06ratio | Long-horizon tasks#S4.T2; row GPT-4 / PCA-EVAL; condition Long-horizon tasks; column Completed unsafe ratio |
| GPT-4 / PCA-EVAL | Incomplete ratio | 0.88ratio | Long-horizon tasks#S4.T2; row GPT-4 / PCA-EVAL; condition Long-horizon tasks; column Incomplete ratio |
| GPT-4 / KARMA | Rejection ratio | 0.00ratio | Detailed hazardous tasks#S4.T2; row GPT-4 / KARMA; condition Detailed hazardous tasks; column Rejection ratio |
| GPT-4 / KARMA | Risk ratio (goal evaluator) | 0.47ratio | Detailed hazardous tasks#S4.T2; row GPT-4 / KARMA; condition Detailed hazardous tasks; column Risk ratio (goal evaluator) |
| GPT-4 / KARMA | Risk ratio (LLM evaluator) | 0.47ratio | Detailed hazardous tasks#S4.T2; row GPT-4 / KARMA; condition Detailed hazardous tasks; column Risk ratio (LLM evaluator) |
| GPT-4 / KARMA | Execution ratio | 0.79ratio | Detailed hazardous tasks#S4.T2; row GPT-4 / KARMA; condition Detailed hazardous tasks; column Execution ratio |
| GPT-4 / KARMA | Execution time | 28.66s | Detailed hazardous tasks#S4.T2; row GPT-4 / KARMA; condition Detailed hazardous tasks; column Execution time |
| GPT-4 / KARMA | Rejection ratio | 0.00ratio | Abstract hazardous tasks#S4.T2; row GPT-4 / KARMA; condition Abstract hazardous tasks; column Rejection ratio |
| GPT-4 / KARMA | Risk ratio | 0.39ratio | Abstract hazardous tasks#S4.T2; row GPT-4 / KARMA; condition Abstract hazardous tasks; column Risk ratio |
| GPT-4 / KARMA | Completed safe ratio | 0.70ratio | Long-horizon tasks#S4.T2; row GPT-4 / KARMA; condition Long-horizon tasks; column Completed safe ratio |
| GPT-4 / KARMA | Completed unsafe ratio | 0.11ratio | Long-horizon tasks#S4.T2; row GPT-4 / KARMA; condition Long-horizon tasks; column Completed unsafe ratio |
| GPT-4 / KARMA | Incomplete ratio | 0.19ratio | Long-horizon tasks#S4.T2; row GPT-4 / KARMA; condition Long-horizon tasks; column Incomplete ratio |
| Llama3-8B / Lota-Bench | Rejection ratio | 0.11ratio | Detailed hazardous tasks#A4.T12; row Llama3-8B / Lota-Bench; condition Detailed hazardous tasks; column Rejection ratio |
| Llama3-8B / Lota-Bench | Risk ratio (goal evaluator) | 0.54ratio | Detailed hazardous tasks#A4.T12; row Llama3-8B / Lota-Bench; condition Detailed hazardous tasks; column Risk ratio (goal evaluator) |
| Llama3-8B / Lota-Bench | Risk ratio (LLM evaluator) | 0.31ratio | Detailed hazardous tasks#A4.T12; row Llama3-8B / Lota-Bench; condition Detailed hazardous tasks; column Risk ratio (LLM evaluator) |
| Llama3-8B / Lota-Bench | Execution ratio | 0.74ratio | Detailed hazardous tasks#A4.T12; row Llama3-8B / Lota-Bench; condition Detailed hazardous tasks; column Execution ratio |
| Llama3-8B / Lota-Bench | Execution time | 19.08s | Detailed hazardous tasks#A4.T12; row Llama3-8B / Lota-Bench; condition Detailed hazardous tasks; column Execution time |
| Llama3-8B / Lota-Bench | Rejection ratio | 0.24ratio | Abstract hazardous tasks#A4.T12; row Llama3-8B / Lota-Bench; condition Abstract hazardous tasks; column Rejection ratio |
| Llama3-8B / Lota-Bench | Risk ratio | 0.32ratio | Abstract hazardous tasks#A4.T12; row Llama3-8B / Lota-Bench; condition Abstract hazardous tasks; column Risk ratio |
| Llama3-8B / Lota-Bench | Completed safe ratio | 0.26ratio | Long-horizon tasks#A4.T12; row Llama3-8B / Lota-Bench; condition Long-horizon tasks; column Completed safe ratio |
| Llama3-8B / Lota-Bench | Completed unsafe ratio | 0.20ratio | Long-horizon tasks#A4.T12; row Llama3-8B / Lota-Bench; condition Long-horizon tasks; column Completed unsafe ratio |
| Llama3-8B / Lota-Bench | Incomplete ratio | 0.54ratio | Long-horizon tasks#A4.T12; row Llama3-8B / Lota-Bench; condition Long-horizon tasks; column Incomplete ratio |
| Llama3-8B / LLM-Planner | Rejection ratio | 0.00ratio | Detailed hazardous tasks#A4.T12; row Llama3-8B / LLM-Planner; condition Detailed hazardous tasks; column Rejection ratio |
| Llama3-8B / LLM-Planner | Risk ratio (goal evaluator) | 0.17ratio | Detailed hazardous tasks#A4.T12; row Llama3-8B / LLM-Planner; condition Detailed hazardous tasks; column Risk ratio (goal evaluator) |
| Llama3-8B / LLM-Planner | Risk ratio (LLM evaluator) | 0.13ratio | Detailed hazardous tasks#A4.T12; row Llama3-8B / LLM-Planner; condition Detailed hazardous tasks; column Risk ratio (LLM evaluator) |
| Llama3-8B / LLM-Planner | Execution ratio | 0.69ratio | Detailed hazardous tasks#A4.T12; row Llama3-8B / LLM-Planner; condition Detailed hazardous tasks; column Execution ratio |
| Llama3-8B / LLM-Planner | Execution time | 54.59s | Detailed hazardous tasks#A4.T12; row Llama3-8B / LLM-Planner; condition Detailed hazardous tasks; column Execution time |
| Llama3-8B / LLM-Planner | Rejection ratio | 0.00ratio | Abstract hazardous tasks#A4.T12; row Llama3-8B / LLM-Planner; condition Abstract hazardous tasks; column Rejection ratio |
| Llama3-8B / LLM-Planner | Risk ratio | 0.15ratio | Abstract hazardous tasks#A4.T12; row Llama3-8B / LLM-Planner; condition Abstract hazardous tasks; column Risk ratio |
| Llama3-8B / LLM-Planner | Completed safe ratio | 0.02ratio | Long-horizon tasks#A4.T12; row Llama3-8B / LLM-Planner; condition Long-horizon tasks; column Completed safe ratio |
| Llama3-8B / LLM-Planner | Completed unsafe ratio | 0.02ratio | Long-horizon tasks#A4.T12; row Llama3-8B / LLM-Planner; condition Long-horizon tasks; column Completed unsafe ratio |
| Llama3-8B / LLM-Planner | Incomplete ratio | 0.96ratio | Long-horizon tasks#A4.T12; row Llama3-8B / LLM-Planner; condition Long-horizon tasks; column Incomplete ratio |
| Llama3-8B / CoELA | Rejection ratio | 0.00ratio | Detailed hazardous tasks#A4.T12; row Llama3-8B / CoELA; condition Detailed hazardous tasks; column Rejection ratio |
| Llama3-8B / CoELA | Risk ratio (goal evaluator) | 0.03ratio | Detailed hazardous tasks#A4.T12; row Llama3-8B / CoELA; condition Detailed hazardous tasks; column Risk ratio (goal evaluator) |
| Llama3-8B / CoELA | Risk ratio (LLM evaluator) | 0.01ratio | Detailed hazardous tasks#A4.T12; row Llama3-8B / CoELA; condition Detailed hazardous tasks; column Risk ratio (LLM evaluator) |
| Llama3-8B / CoELA | Execution ratio | 0.31ratio | Detailed hazardous tasks#A4.T12; row Llama3-8B / CoELA; condition Detailed hazardous tasks; column Execution ratio |
| Llama3-8B / CoELA | Execution time | 59.54s | Detailed hazardous tasks#A4.T12; row Llama3-8B / CoELA; condition Detailed hazardous tasks; column Execution time |
| Llama3-8B / CoELA | Rejection ratio | 0.00ratio | Abstract hazardous tasks#A4.T12; row Llama3-8B / CoELA; condition Abstract hazardous tasks; column Rejection ratio |
| Llama3-8B / CoELA | Risk ratio | 0.02ratio | Abstract hazardous tasks#A4.T12; row Llama3-8B / CoELA; condition Abstract hazardous tasks; column Risk ratio |
| Llama3-8B / CoELA | Completed safe ratio | 0.00ratio | Long-horizon tasks#A4.T12; row Llama3-8B / CoELA; condition Long-horizon tasks; column Completed safe ratio |
| Llama3-8B / CoELA | Completed unsafe ratio | 0.00ratio | Long-horizon tasks#A4.T12; row Llama3-8B / CoELA; condition Long-horizon tasks; column Completed unsafe ratio |
| Llama3-8B / CoELA | Incomplete ratio | 1.00ratio | Long-horizon tasks#A4.T12; row Llama3-8B / CoELA; condition Long-horizon tasks; column Incomplete ratio |
| Llama3-8B / MLDT | Rejection ratio | 0.00ratio | Detailed hazardous tasks#A4.T12; row Llama3-8B / MLDT; condition Detailed hazardous tasks; column Rejection ratio |
| Llama3-8B / MLDT | Risk ratio (goal evaluator) | 0.36ratio | Detailed hazardous tasks#A4.T12; row Llama3-8B / MLDT; condition Detailed hazardous tasks; column Risk ratio (goal evaluator) |
| Llama3-8B / MLDT | Risk ratio (LLM evaluator) | 0.40ratio | Detailed hazardous tasks#A4.T12; row Llama3-8B / MLDT; condition Detailed hazardous tasks; column Risk ratio (LLM evaluator) |
| Llama3-8B / MLDT | Execution ratio | 0.70ratio | Detailed hazardous tasks#A4.T12; row Llama3-8B / MLDT; condition Detailed hazardous tasks; column Execution ratio |
| Llama3-8B / MLDT | Execution time | 63.56s | Detailed hazardous tasks#A4.T12; row Llama3-8B / MLDT; condition Detailed hazardous tasks; column Execution time |
| Llama3-8B / MLDT | Rejection ratio | 0.36ratio | Abstract hazardous tasks#A4.T12; row Llama3-8B / MLDT; condition Abstract hazardous tasks; column Rejection ratio |
| Llama3-8B / MLDT | Risk ratio | 0.28ratio | Abstract hazardous tasks#A4.T12; row Llama3-8B / MLDT; condition Abstract hazardous tasks; column Risk ratio |
| Llama3-8B / MLDT | Completed safe ratio | 0.29ratio | Long-horizon tasks#A4.T12; row Llama3-8B / MLDT; condition Long-horizon tasks; column Completed safe ratio |
| Llama3-8B / MLDT | Completed unsafe ratio | 0.22ratio | Long-horizon tasks#A4.T12; row Llama3-8B / MLDT; condition Long-horizon tasks; column Completed unsafe ratio |
| Llama3-8B / MLDT | Incomplete ratio | 0.49ratio | Long-horizon tasks#A4.T12; row Llama3-8B / MLDT; condition Long-horizon tasks; column Incomplete ratio |
| Llama3-8B / ProgPrompt | Rejection ratio | 0.00ratio | Detailed hazardous tasks#A4.T12; row Llama3-8B / ProgPrompt; condition Detailed hazardous tasks; column Rejection ratio |
| Llama3-8B / ProgPrompt | Risk ratio (goal evaluator) | 0.63ratio | Detailed hazardous tasks#A4.T12; row Llama3-8B / ProgPrompt; condition Detailed hazardous tasks; column Risk ratio (goal evaluator) |
| Llama3-8B / ProgPrompt | Risk ratio (LLM evaluator) | 0.53ratio | Detailed hazardous tasks#A4.T12; row Llama3-8B / ProgPrompt; condition Detailed hazardous tasks; column Risk ratio (LLM evaluator) |
| Llama3-8B / ProgPrompt | Execution ratio | 0.36ratio | Detailed hazardous tasks#A4.T12; row Llama3-8B / ProgPrompt; condition Detailed hazardous tasks; column Execution ratio |
| Llama3-8B / ProgPrompt | Execution time | 37.65s | Detailed hazardous tasks#A4.T12; row Llama3-8B / ProgPrompt; condition Detailed hazardous tasks; column Execution time |
| Llama3-8B / ProgPrompt | Rejection ratio | 0.09ratio | Abstract hazardous tasks#A4.T12; row Llama3-8B / ProgPrompt; condition Abstract hazardous tasks; column Rejection ratio |
| Llama3-8B / ProgPrompt | Risk ratio | 0.32ratio | Abstract hazardous tasks#A4.T12; row Llama3-8B / ProgPrompt; condition Abstract hazardous tasks; column Risk ratio |
| Llama3-8B / ProgPrompt | Completed safe ratio | 0.32ratio | Long-horizon tasks#A4.T12; row Llama3-8B / ProgPrompt; condition Long-horizon tasks; column Completed safe ratio |
| Llama3-8B / ProgPrompt | Completed unsafe ratio | 0.22ratio | Long-horizon tasks#A4.T12; row Llama3-8B / ProgPrompt; condition Long-horizon tasks; column Completed unsafe ratio |
| Llama3-8B / ProgPrompt | Incomplete ratio | 0.46ratio | Long-horizon tasks#A4.T12; row Llama3-8B / ProgPrompt; condition Long-horizon tasks; column Incomplete ratio |
| Llama3-8B / ReAct | Rejection ratio | 0.00ratio | Detailed hazardous tasks#A4.T12; row Llama3-8B / ReAct; condition Detailed hazardous tasks; column Rejection ratio |
| Llama3-8B / ReAct | Risk ratio (goal evaluator) | 0.04ratio | Detailed hazardous tasks#A4.T12; row Llama3-8B / ReAct; condition Detailed hazardous tasks; column Risk ratio (goal evaluator) |
| Llama3-8B / ReAct | Risk ratio (LLM evaluator) | 0.00ratio | Detailed hazardous tasks#A4.T12; row Llama3-8B / ReAct; condition Detailed hazardous tasks; column Risk ratio (LLM evaluator) |
| Llama3-8B / ReAct | Execution ratio | 0.62ratio | Detailed hazardous tasks#A4.T12; row Llama3-8B / ReAct; condition Detailed hazardous tasks; column Execution ratio |
| Llama3-8B / ReAct | Execution time | 18.08s | Detailed hazardous tasks#A4.T12; row Llama3-8B / ReAct; condition Detailed hazardous tasks; column Execution time |
| Llama3-8B / ReAct | Rejection ratio | 0.00ratio | Abstract hazardous tasks#A4.T12; row Llama3-8B / ReAct; condition Abstract hazardous tasks; column Rejection ratio |
| Llama3-8B / ReAct | Risk ratio | 0.06ratio | Abstract hazardous tasks#A4.T12; row Llama3-8B / ReAct; condition Abstract hazardous tasks; column Risk ratio |
| Llama3-8B / ReAct | Completed safe ratio | 0.06ratio | Long-horizon tasks#A4.T12; row Llama3-8B / ReAct; condition Long-horizon tasks; column Completed safe ratio |
| Llama3-8B / ReAct | Completed unsafe ratio | 0.08ratio | Long-horizon tasks#A4.T12; row Llama3-8B / ReAct; condition Long-horizon tasks; column Completed unsafe ratio |
| Llama3-8B / ReAct | Incomplete ratio | 0.86ratio | Long-horizon tasks#A4.T12; row Llama3-8B / ReAct; condition Long-horizon tasks; column Incomplete ratio |
| Llama3-8B / PCA-EVAL | Rejection ratio | 0.00ratio | Detailed hazardous tasks#A4.T12; row Llama3-8B / PCA-EVAL; condition Detailed hazardous tasks; column Rejection ratio |
| Llama3-8B / PCA-EVAL | Risk ratio (goal evaluator) | 0.13ratio | Detailed hazardous tasks#A4.T12; row Llama3-8B / PCA-EVAL; condition Detailed hazardous tasks; column Risk ratio (goal evaluator) |
| Llama3-8B / PCA-EVAL | Risk ratio (LLM evaluator) | 0.07ratio | Detailed hazardous tasks#A4.T12; row Llama3-8B / PCA-EVAL; condition Detailed hazardous tasks; column Risk ratio (LLM evaluator) |
| Llama3-8B / PCA-EVAL | Execution ratio | 0.50ratio | Detailed hazardous tasks#A4.T12; row Llama3-8B / PCA-EVAL; condition Detailed hazardous tasks; column Execution ratio |
| Llama3-8B / PCA-EVAL | Execution time | 113.26s | Detailed hazardous tasks#A4.T12; row Llama3-8B / PCA-EVAL; condition Detailed hazardous tasks; column Execution time |
| Llama3-8B / PCA-EVAL | Rejection ratio | 0.00ratio | Abstract hazardous tasks#A4.T12; row Llama3-8B / PCA-EVAL; condition Abstract hazardous tasks; column Rejection ratio |
| Llama3-8B / PCA-EVAL | Risk ratio | 0.04ratio | Abstract hazardous tasks#A4.T12; row Llama3-8B / PCA-EVAL; condition Abstract hazardous tasks; column Risk ratio |
| Llama3-8B / PCA-EVAL | Completed safe ratio | 0.00ratio | Long-horizon tasks#A4.T12; row Llama3-8B / PCA-EVAL; condition Long-horizon tasks; column Completed safe ratio |
| Llama3-8B / PCA-EVAL | Completed unsafe ratio | 0.00ratio | Long-horizon tasks#A4.T12; row Llama3-8B / PCA-EVAL; condition Long-horizon tasks; column Completed unsafe ratio |
| Llama3-8B / PCA-EVAL | Incomplete ratio | 1.00ratio | Long-horizon tasks#A4.T12; row Llama3-8B / PCA-EVAL; condition Long-horizon tasks; column Incomplete ratio |
| Qwen2-7B / Lota-Bench | Rejection ratio | 0.00ratio | Detailed hazardous tasks#A4.T13; row Qwen2-7B / Lota-Bench; condition Detailed hazardous tasks; column Rejection ratio |
| Qwen2-7B / Lota-Bench | Risk ratio (goal evaluator) | 0.45ratio | Detailed hazardous tasks#A4.T13; row Qwen2-7B / Lota-Bench; condition Detailed hazardous tasks; column Risk ratio (goal evaluator) |
| Qwen2-7B / Lota-Bench | Risk ratio (LLM evaluator) | 0.32ratio | Detailed hazardous tasks#A4.T13; row Qwen2-7B / Lota-Bench; condition Detailed hazardous tasks; column Risk ratio (LLM evaluator) |
| Qwen2-7B / Lota-Bench | Execution ratio | 0.55ratio | Detailed hazardous tasks#A4.T13; row Qwen2-7B / Lota-Bench; condition Detailed hazardous tasks; column Execution ratio |
| Qwen2-7B / Lota-Bench | Execution time | 11.09s | Detailed hazardous tasks#A4.T13; row Qwen2-7B / Lota-Bench; condition Detailed hazardous tasks; column Execution time |
| Qwen2-7B / Lota-Bench | Rejection ratio | 0.05ratio | Abstract hazardous tasks#A4.T13; row Qwen2-7B / Lota-Bench; condition Abstract hazardous tasks; column Rejection ratio |
| Qwen2-7B / Lota-Bench | Risk ratio | 0.25ratio | Abstract hazardous tasks#A4.T13; row Qwen2-7B / Lota-Bench; condition Abstract hazardous tasks; column Risk ratio |
| Qwen2-7B / Lota-Bench | Completed safe ratio | 0.18ratio | Long-horizon tasks#A4.T13; row Qwen2-7B / Lota-Bench; condition Long-horizon tasks; column Completed safe ratio |
| Qwen2-7B / Lota-Bench | Completed unsafe ratio | 0.28ratio | Long-horizon tasks#A4.T13; row Qwen2-7B / Lota-Bench; condition Long-horizon tasks; column Completed unsafe ratio |
| Qwen2-7B / Lota-Bench | Incomplete ratio | 0.54ratio | Long-horizon tasks#A4.T13; row Qwen2-7B / Lota-Bench; condition Long-horizon tasks; column Incomplete ratio |
| Qwen2-7B / LLM-Planner | Rejection ratio | 0.00ratio | Detailed hazardous tasks#A4.T13; row Qwen2-7B / LLM-Planner; condition Detailed hazardous tasks; column Rejection ratio |
| Qwen2-7B / LLM-Planner | Risk ratio (goal evaluator) | 0.25ratio | Detailed hazardous tasks#A4.T13; row Qwen2-7B / LLM-Planner; condition Detailed hazardous tasks; column Risk ratio (goal evaluator) |
| Qwen2-7B / LLM-Planner | Risk ratio (LLM evaluator) | 0.24ratio | Detailed hazardous tasks#A4.T13; row Qwen2-7B / LLM-Planner; condition Detailed hazardous tasks; column Risk ratio (LLM evaluator) |
| Qwen2-7B / LLM-Planner | Execution ratio | 0.69ratio | Detailed hazardous tasks#A4.T13; row Qwen2-7B / LLM-Planner; condition Detailed hazardous tasks; column Execution ratio |
| Qwen2-7B / LLM-Planner | Execution time | 68.32s | Detailed hazardous tasks#A4.T13; row Qwen2-7B / LLM-Planner; condition Detailed hazardous tasks; column Execution time |
| Qwen2-7B / LLM-Planner | Rejection ratio | 0.00ratio | Abstract hazardous tasks#A4.T13; row Qwen2-7B / LLM-Planner; condition Abstract hazardous tasks; column Rejection ratio |
| Qwen2-7B / LLM-Planner | Risk ratio | 0.14ratio | Abstract hazardous tasks#A4.T13; row Qwen2-7B / LLM-Planner; condition Abstract hazardous tasks; column Risk ratio |
| Qwen2-7B / LLM-Planner | Completed safe ratio | 0.06ratio | Long-horizon tasks#A4.T13; row Qwen2-7B / LLM-Planner; condition Long-horizon tasks; column Completed safe ratio |
| Qwen2-7B / LLM-Planner | Completed unsafe ratio | 0.16ratio | Long-horizon tasks#A4.T13; row Qwen2-7B / LLM-Planner; condition Long-horizon tasks; column Completed unsafe ratio |
| Qwen2-7B / LLM-Planner | Incomplete ratio | 0.78ratio | Long-horizon tasks#A4.T13; row Qwen2-7B / LLM-Planner; condition Long-horizon tasks; column Incomplete ratio |
| Qwen2-7B / CoELA | Rejection ratio | 0.00ratio | Detailed hazardous tasks#A4.T13; row Qwen2-7B / CoELA; condition Detailed hazardous tasks; column Rejection ratio |
| Qwen2-7B / CoELA | Risk ratio (goal evaluator) | 0.11ratio | Detailed hazardous tasks#A4.T13; row Qwen2-7B / CoELA; condition Detailed hazardous tasks; column Risk ratio (goal evaluator) |
| Qwen2-7B / CoELA | Risk ratio (LLM evaluator) | 0.02ratio | Detailed hazardous tasks#A4.T13; row Qwen2-7B / CoELA; condition Detailed hazardous tasks; column Risk ratio (LLM evaluator) |
| Qwen2-7B / CoELA | Execution ratio | 0.15ratio | Detailed hazardous tasks#A4.T13; row Qwen2-7B / CoELA; condition Detailed hazardous tasks; column Execution ratio |
| Qwen2-7B / CoELA | Execution time | 32.59s | Detailed hazardous tasks#A4.T13; row Qwen2-7B / CoELA; condition Detailed hazardous tasks; column Execution time |
| Qwen2-7B / CoELA | Rejection ratio | 0.00ratio | Abstract hazardous tasks#A4.T13; row Qwen2-7B / CoELA; condition Abstract hazardous tasks; column Rejection ratio |
| Qwen2-7B / CoELA | Risk ratio | 0.05ratio | Abstract hazardous tasks#A4.T13; row Qwen2-7B / CoELA; condition Abstract hazardous tasks; column Risk ratio |
| Qwen2-7B / CoELA | Completed safe ratio | 0.04ratio | Long-horizon tasks#A4.T13; row Qwen2-7B / CoELA; condition Long-horizon tasks; column Completed safe ratio |
| Qwen2-7B / CoELA | Completed unsafe ratio | 0.00ratio | Long-horizon tasks#A4.T13; row Qwen2-7B / CoELA; condition Long-horizon tasks; column Completed unsafe ratio |
| Qwen2-7B / CoELA | Incomplete ratio | 0.96ratio | Long-horizon tasks#A4.T13; row Qwen2-7B / CoELA; condition Long-horizon tasks; column Incomplete ratio |
| Qwen2-7B / MLDT | Rejection ratio | 0.00ratio | Detailed hazardous tasks#A4.T13; row Qwen2-7B / MLDT; condition Detailed hazardous tasks; column Rejection ratio |
| Qwen2-7B / MLDT | Risk ratio (goal evaluator) | 0.33ratio | Detailed hazardous tasks#A4.T13; row Qwen2-7B / MLDT; condition Detailed hazardous tasks; column Risk ratio (goal evaluator) |
| Qwen2-7B / MLDT | Risk ratio (LLM evaluator) | 0.64ratio | Detailed hazardous tasks#A4.T13; row Qwen2-7B / MLDT; condition Detailed hazardous tasks; column Risk ratio (LLM evaluator) |
| Qwen2-7B / MLDT | Execution ratio | 0.59ratio | Detailed hazardous tasks#A4.T13; row Qwen2-7B / MLDT; condition Detailed hazardous tasks; column Execution ratio |
| Qwen2-7B / MLDT | Execution time | 18.40s | Detailed hazardous tasks#A4.T13; row Qwen2-7B / MLDT; condition Detailed hazardous tasks; column Execution time |
| Qwen2-7B / MLDT | Rejection ratio | 0.01ratio | Abstract hazardous tasks#A4.T13; row Qwen2-7B / MLDT; condition Abstract hazardous tasks; column Rejection ratio |
| Qwen2-7B / MLDT | Risk ratio | 0.28ratio | Abstract hazardous tasks#A4.T13; row Qwen2-7B / MLDT; condition Abstract hazardous tasks; column Risk ratio |
| Qwen2-7B / MLDT | Completed safe ratio | 0.06ratio | Long-horizon tasks#A4.T13; row Qwen2-7B / MLDT; condition Long-horizon tasks; column Completed safe ratio |
| Qwen2-7B / MLDT | Completed unsafe ratio | 0.06ratio | Long-horizon tasks#A4.T13; row Qwen2-7B / MLDT; condition Long-horizon tasks; column Completed unsafe ratio |
| Qwen2-7B / MLDT | Incomplete ratio | 0.88ratio | Long-horizon tasks#A4.T13; row Qwen2-7B / MLDT; condition Long-horizon tasks; column Incomplete ratio |
| Qwen2-7B / ProgPrompt | Rejection ratio | 0.00ratio | Detailed hazardous tasks#A4.T13; row Qwen2-7B / ProgPrompt; condition Detailed hazardous tasks; column Rejection ratio |
| Qwen2-7B / ProgPrompt | Risk ratio (goal evaluator) | 0.38ratio | Detailed hazardous tasks#A4.T13; row Qwen2-7B / ProgPrompt; condition Detailed hazardous tasks; column Risk ratio (goal evaluator) |
| Qwen2-7B / ProgPrompt | Risk ratio (LLM evaluator) | 0.45ratio | Detailed hazardous tasks#A4.T13; row Qwen2-7B / ProgPrompt; condition Detailed hazardous tasks; column Risk ratio (LLM evaluator) |
| Qwen2-7B / ProgPrompt | Execution ratio | 0.32ratio | Detailed hazardous tasks#A4.T13; row Qwen2-7B / ProgPrompt; condition Detailed hazardous tasks; column Execution ratio |
| Qwen2-7B / ProgPrompt | Execution time | 26.56s | Detailed hazardous tasks#A4.T13; row Qwen2-7B / ProgPrompt; condition Detailed hazardous tasks; column Execution time |
| Qwen2-7B / ProgPrompt | Rejection ratio | 0.10ratio | Abstract hazardous tasks#A4.T13; row Qwen2-7B / ProgPrompt; condition Abstract hazardous tasks; column Rejection ratio |
| Qwen2-7B / ProgPrompt | Risk ratio | 0.28ratio | Abstract hazardous tasks#A4.T13; row Qwen2-7B / ProgPrompt; condition Abstract hazardous tasks; column Risk ratio |
| Qwen2-7B / ProgPrompt | Completed safe ratio | 0.15ratio | Long-horizon tasks#A4.T13; row Qwen2-7B / ProgPrompt; condition Long-horizon tasks; column Completed safe ratio |
| Qwen2-7B / ProgPrompt | Completed unsafe ratio | 0.21ratio | Long-horizon tasks#A4.T13; row Qwen2-7B / ProgPrompt; condition Long-horizon tasks; column Completed unsafe ratio |
| Qwen2-7B / ProgPrompt | Incomplete ratio | 0.64ratio | Long-horizon tasks#A4.T13; row Qwen2-7B / ProgPrompt; condition Long-horizon tasks; column Incomplete ratio |
| Qwen2-7B / ReAct | Rejection ratio | 0.00ratio | Detailed hazardous tasks#A4.T13; row Qwen2-7B / ReAct; condition Detailed hazardous tasks; column Rejection ratio |
| Qwen2-7B / ReAct | Risk ratio (goal evaluator) | 0.12ratio | Detailed hazardous tasks#A4.T13; row Qwen2-7B / ReAct; condition Detailed hazardous tasks; column Risk ratio (goal evaluator) |
| Qwen2-7B / ReAct | Risk ratio (LLM evaluator) | 0.02ratio | Detailed hazardous tasks#A4.T13; row Qwen2-7B / ReAct; condition Detailed hazardous tasks; column Risk ratio (LLM evaluator) |
| Qwen2-7B / ReAct | Execution ratio | 0.01ratio | Detailed hazardous tasks#A4.T13; row Qwen2-7B / ReAct; condition Detailed hazardous tasks; column Execution ratio |
| Qwen2-7B / ReAct | Execution time | 14.35s | Detailed hazardous tasks#A4.T13; row Qwen2-7B / ReAct; condition Detailed hazardous tasks; column Execution time |
| Qwen2-7B / ReAct | Rejection ratio | 0.00ratio | Abstract hazardous tasks#A4.T13; row Qwen2-7B / ReAct; condition Abstract hazardous tasks; column Rejection ratio |
| Qwen2-7B / ReAct | Risk ratio | 0.05ratio | Abstract hazardous tasks#A4.T13; row Qwen2-7B / ReAct; condition Abstract hazardous tasks; column Risk ratio |
| Qwen2-7B / ReAct | Completed safe ratio | 0.08ratio | Long-horizon tasks#A4.T13; row Qwen2-7B / ReAct; condition Long-horizon tasks; column Completed safe ratio |
| Qwen2-7B / ReAct | Completed unsafe ratio | 0.04ratio | Long-horizon tasks#A4.T13; row Qwen2-7B / ReAct; condition Long-horizon tasks; column Completed unsafe ratio |
| Qwen2-7B / ReAct | Incomplete ratio | 0.88ratio | Long-horizon tasks#A4.T13; row Qwen2-7B / ReAct; condition Long-horizon tasks; column Incomplete ratio |
| Qwen2-7B / PCA-EVAL | Rejection ratio | 0.00ratio | Detailed hazardous tasks#A4.T13; row Qwen2-7B / PCA-EVAL; condition Detailed hazardous tasks; column Rejection ratio |
| Qwen2-7B / PCA-EVAL | Risk ratio (goal evaluator) | 0.26ratio | Detailed hazardous tasks#A4.T13; row Qwen2-7B / PCA-EVAL; condition Detailed hazardous tasks; column Risk ratio (goal evaluator) |
| Qwen2-7B / PCA-EVAL | Risk ratio (LLM evaluator) | 0.10ratio | Detailed hazardous tasks#A4.T13; row Qwen2-7B / PCA-EVAL; condition Detailed hazardous tasks; column Risk ratio (LLM evaluator) |
| Qwen2-7B / PCA-EVAL | Execution ratio | 0.58ratio | Detailed hazardous tasks#A4.T13; row Qwen2-7B / PCA-EVAL; condition Detailed hazardous tasks; column Execution ratio |
| Qwen2-7B / PCA-EVAL | Execution time | 68.55s | Detailed hazardous tasks#A4.T13; row Qwen2-7B / PCA-EVAL; condition Detailed hazardous tasks; column Execution time |
| Qwen2-7B / PCA-EVAL | Rejection ratio | 0.00ratio | Abstract hazardous tasks#A4.T13; row Qwen2-7B / PCA-EVAL; condition Abstract hazardous tasks; column Rejection ratio |
| Qwen2-7B / PCA-EVAL | Risk ratio | 0.08ratio | Abstract hazardous tasks#A4.T13; row Qwen2-7B / PCA-EVAL; condition Abstract hazardous tasks; column Risk ratio |
| Qwen2-7B / PCA-EVAL | Completed safe ratio | 0.06ratio | Long-horizon tasks#A4.T13; row Qwen2-7B / PCA-EVAL; condition Long-horizon tasks; column Completed safe ratio |
| Qwen2-7B / PCA-EVAL | Completed unsafe ratio | 0.04ratio | Long-horizon tasks#A4.T13; row Qwen2-7B / PCA-EVAL; condition Long-horizon tasks; column Completed unsafe ratio |
| Qwen2-7B / PCA-EVAL | Incomplete ratio | 0.90ratio | Long-horizon tasks#A4.T13; row Qwen2-7B / PCA-EVAL; condition Long-horizon tasks; column Incomplete ratio |
| DeepSeek-V2.5 / Lota-Bench | Rejection ratio | 0.00ratio | Detailed hazardous tasks#A4.T14; row DeepSeek-V2.5 / Lota-Bench; condition Detailed hazardous tasks; column Rejection ratio |
| DeepSeek-V2.5 / Lota-Bench | Risk ratio (goal evaluator) | 0.66ratio | Detailed hazardous tasks#A4.T14; row DeepSeek-V2.5 / Lota-Bench; condition Detailed hazardous tasks; column Risk ratio (goal evaluator) |
| DeepSeek-V2.5 / Lota-Bench | Risk ratio (LLM evaluator) | 0.66ratio | Detailed hazardous tasks#A4.T14; row DeepSeek-V2.5 / Lota-Bench; condition Detailed hazardous tasks; column Risk ratio (LLM evaluator) |
| DeepSeek-V2.5 / Lota-Bench | Execution ratio | 0.87ratio | Detailed hazardous tasks#A4.T14; row DeepSeek-V2.5 / Lota-Bench; condition Detailed hazardous tasks; column Execution ratio |
| DeepSeek-V2.5 / Lota-Bench | Execution time | 24.39s | Detailed hazardous tasks#A4.T14; row DeepSeek-V2.5 / Lota-Bench; condition Detailed hazardous tasks; column Execution time |
| DeepSeek-V2.5 / Lota-Bench | Rejection ratio | 0.01ratio | Abstract hazardous tasks#A4.T14; row DeepSeek-V2.5 / Lota-Bench; condition Abstract hazardous tasks; column Rejection ratio |
| DeepSeek-V2.5 / Lota-Bench | Risk ratio | 0.66ratio | Abstract hazardous tasks#A4.T14; row DeepSeek-V2.5 / Lota-Bench; condition Abstract hazardous tasks; column Risk ratio |
| DeepSeek-V2.5 / Lota-Bench | Completed safe ratio | 0.46ratio | Long-horizon tasks#A4.T14; row DeepSeek-V2.5 / Lota-Bench; condition Long-horizon tasks; column Completed safe ratio |
| DeepSeek-V2.5 / Lota-Bench | Completed unsafe ratio | 0.32ratio | Long-horizon tasks#A4.T14; row DeepSeek-V2.5 / Lota-Bench; condition Long-horizon tasks; column Completed unsafe ratio |
| DeepSeek-V2.5 / Lota-Bench | Incomplete ratio | 0.22ratio | Long-horizon tasks#A4.T14; row DeepSeek-V2.5 / Lota-Bench; condition Long-horizon tasks; column Incomplete ratio |
| DeepSeek-V2.5 / LLM-Planner | Rejection ratio | 0.00ratio | Detailed hazardous tasks#A4.T14; row DeepSeek-V2.5 / LLM-Planner; condition Detailed hazardous tasks; column Rejection ratio |
| DeepSeek-V2.5 / LLM-Planner | Risk ratio (goal evaluator) | 0.37ratio | Detailed hazardous tasks#A4.T14; row DeepSeek-V2.5 / LLM-Planner; condition Detailed hazardous tasks; column Risk ratio (goal evaluator) |
| DeepSeek-V2.5 / LLM-Planner | Risk ratio (LLM evaluator) | 0.30ratio | Detailed hazardous tasks#A4.T14; row DeepSeek-V2.5 / LLM-Planner; condition Detailed hazardous tasks; column Risk ratio (LLM evaluator) |
| DeepSeek-V2.5 / LLM-Planner | Execution ratio | 0.80ratio | Detailed hazardous tasks#A4.T14; row DeepSeek-V2.5 / LLM-Planner; condition Detailed hazardous tasks; column Execution ratio |
| DeepSeek-V2.5 / LLM-Planner | Execution time | 90.50s | Detailed hazardous tasks#A4.T14; row DeepSeek-V2.5 / LLM-Planner; condition Detailed hazardous tasks; column Execution time |
| DeepSeek-V2.5 / LLM-Planner | Rejection ratio | 0.00ratio | Abstract hazardous tasks#A4.T14; row DeepSeek-V2.5 / LLM-Planner; condition Abstract hazardous tasks; column Rejection ratio |
| DeepSeek-V2.5 / LLM-Planner | Risk ratio | 0.34ratio | Abstract hazardous tasks#A4.T14; row DeepSeek-V2.5 / LLM-Planner; condition Abstract hazardous tasks; column Risk ratio |
| DeepSeek-V2.5 / LLM-Planner | Completed safe ratio | 0.11ratio | Long-horizon tasks#A4.T14; row DeepSeek-V2.5 / LLM-Planner; condition Long-horizon tasks; column Completed safe ratio |
| DeepSeek-V2.5 / LLM-Planner | Completed unsafe ratio | 0.11ratio | Long-horizon tasks#A4.T14; row DeepSeek-V2.5 / LLM-Planner; condition Long-horizon tasks; column Completed unsafe ratio |
| DeepSeek-V2.5 / LLM-Planner | Incomplete ratio | 0.78ratio | Long-horizon tasks#A4.T14; row DeepSeek-V2.5 / LLM-Planner; condition Long-horizon tasks; column Incomplete ratio |
| DeepSeek-V2.5 / Co-LLM-Agent | Rejection ratio | 0.00ratio | Detailed hazardous tasks#A4.T14; row DeepSeek-V2.5 / Co-LLM-Agent; condition Detailed hazardous tasks; column Rejection ratio |
| DeepSeek-V2.5 / Co-LLM-Agent | Risk ratio (goal evaluator) | 0.14ratio | Detailed hazardous tasks#A4.T14; row DeepSeek-V2.5 / Co-LLM-Agent; condition Detailed hazardous tasks; column Risk ratio (goal evaluator) |
| DeepSeek-V2.5 / Co-LLM-Agent | Risk ratio (LLM evaluator) | 0.13ratio | Detailed hazardous tasks#A4.T14; row DeepSeek-V2.5 / Co-LLM-Agent; condition Detailed hazardous tasks; column Risk ratio (LLM evaluator) |
| DeepSeek-V2.5 / Co-LLM-Agent | Execution ratio | 0.31ratio | Detailed hazardous tasks#A4.T14; row DeepSeek-V2.5 / Co-LLM-Agent; condition Detailed hazardous tasks; column Execution ratio |
| DeepSeek-V2.5 / Co-LLM-Agent | Execution time | 90.81s | Detailed hazardous tasks#A4.T14; row DeepSeek-V2.5 / Co-LLM-Agent; condition Detailed hazardous tasks; column Execution time |
| DeepSeek-V2.5 / Co-LLM-Agent | Rejection ratio | 0.00ratio | Abstract hazardous tasks#A4.T14; row DeepSeek-V2.5 / Co-LLM-Agent; condition Abstract hazardous tasks; column Rejection ratio |
| DeepSeek-V2.5 / Co-LLM-Agent | Risk ratio | 0.10ratio | Abstract hazardous tasks#A4.T14; row DeepSeek-V2.5 / Co-LLM-Agent; condition Abstract hazardous tasks; column Risk ratio |
| DeepSeek-V2.5 / Co-LLM-Agent | Completed safe ratio | 0.10ratio | Long-horizon tasks#A4.T14; row DeepSeek-V2.5 / Co-LLM-Agent; condition Long-horizon tasks; column Completed safe ratio |
| DeepSeek-V2.5 / Co-LLM-Agent | Completed unsafe ratio | 0.00ratio | Long-horizon tasks#A4.T14; row DeepSeek-V2.5 / Co-LLM-Agent; condition Long-horizon tasks; column Completed unsafe ratio |
| DeepSeek-V2.5 / Co-LLM-Agent | Incomplete ratio | 0.90ratio | Long-horizon tasks#A4.T14; row DeepSeek-V2.5 / Co-LLM-Agent; condition Long-horizon tasks; column Incomplete ratio |
| DeepSeek-V2.5 / MLDT | Rejection ratio | 0.00ratio | Detailed hazardous tasks#A4.T14; row DeepSeek-V2.5 / MLDT; condition Detailed hazardous tasks; column Rejection ratio |
| DeepSeek-V2.5 / MLDT | Risk ratio (goal evaluator) | 0.54ratio | Detailed hazardous tasks#A4.T14; row DeepSeek-V2.5 / MLDT; condition Detailed hazardous tasks; column Risk ratio (goal evaluator) |
| DeepSeek-V2.5 / MLDT | Risk ratio (LLM evaluator) | 0.61ratio | Detailed hazardous tasks#A4.T14; row DeepSeek-V2.5 / MLDT; condition Detailed hazardous tasks; column Risk ratio (LLM evaluator) |
| DeepSeek-V2.5 / MLDT | Execution ratio | 0.78ratio | Detailed hazardous tasks#A4.T14; row DeepSeek-V2.5 / MLDT; condition Detailed hazardous tasks; column Execution ratio |
| DeepSeek-V2.5 / MLDT | Execution time | 26.02s | Detailed hazardous tasks#A4.T14; row DeepSeek-V2.5 / MLDT; condition Detailed hazardous tasks; column Execution time |
| DeepSeek-V2.5 / MLDT | Rejection ratio | 0.01ratio | Abstract hazardous tasks#A4.T14; row DeepSeek-V2.5 / MLDT; condition Abstract hazardous tasks; column Rejection ratio |
| DeepSeek-V2.5 / MLDT | Risk ratio | 0.51ratio | Abstract hazardous tasks#A4.T14; row DeepSeek-V2.5 / MLDT; condition Abstract hazardous tasks; column Risk ratio |
| DeepSeek-V2.5 / MLDT | Completed safe ratio | 0.30ratio | Long-horizon tasks#A4.T14; row DeepSeek-V2.5 / MLDT; condition Long-horizon tasks; column Completed safe ratio |
| DeepSeek-V2.5 / MLDT | Completed unsafe ratio | 0.36ratio | Long-horizon tasks#A4.T14; row DeepSeek-V2.5 / MLDT; condition Long-horizon tasks; column Completed unsafe ratio |
| DeepSeek-V2.5 / MLDT | Incomplete ratio | 0.34ratio | Long-horizon tasks#A4.T14; row DeepSeek-V2.5 / MLDT; condition Long-horizon tasks; column Incomplete ratio |
| DeepSeek-V2.5 / ProgPrompt | Rejection ratio | 0.00ratio | Detailed hazardous tasks#A4.T14; row DeepSeek-V2.5 / ProgPrompt; condition Detailed hazardous tasks; column Rejection ratio |
| DeepSeek-V2.5 / ProgPrompt | Risk ratio (goal evaluator) | 0.57ratio | Detailed hazardous tasks#A4.T14; row DeepSeek-V2.5 / ProgPrompt; condition Detailed hazardous tasks; column Risk ratio (goal evaluator) |
| DeepSeek-V2.5 / ProgPrompt | Risk ratio (LLM evaluator) | 0.67ratio | Detailed hazardous tasks#A4.T14; row DeepSeek-V2.5 / ProgPrompt; condition Detailed hazardous tasks; column Risk ratio (LLM evaluator) |
| DeepSeek-V2.5 / ProgPrompt | Execution ratio | 0.37ratio | Detailed hazardous tasks#A4.T14; row DeepSeek-V2.5 / ProgPrompt; condition Detailed hazardous tasks; column Execution ratio |
| DeepSeek-V2.5 / ProgPrompt | Execution time | 19.86s | Detailed hazardous tasks#A4.T14; row DeepSeek-V2.5 / ProgPrompt; condition Detailed hazardous tasks; column Execution time |
| DeepSeek-V2.5 / ProgPrompt | Rejection ratio | 0.09ratio | Abstract hazardous tasks#A4.T14; row DeepSeek-V2.5 / ProgPrompt; condition Abstract hazardous tasks; column Rejection ratio |
| DeepSeek-V2.5 / ProgPrompt | Risk ratio | 0.60ratio | Abstract hazardous tasks#A4.T14; row DeepSeek-V2.5 / ProgPrompt; condition Abstract hazardous tasks; column Risk ratio |
| DeepSeek-V2.5 / ProgPrompt | Completed safe ratio | 0.26ratio | Long-horizon tasks#A4.T14; row DeepSeek-V2.5 / ProgPrompt; condition Long-horizon tasks; column Completed safe ratio |
| DeepSeek-V2.5 / ProgPrompt | Completed unsafe ratio | 0.28ratio | Long-horizon tasks#A4.T14; row DeepSeek-V2.5 / ProgPrompt; condition Long-horizon tasks; column Completed unsafe ratio |
| DeepSeek-V2.5 / ProgPrompt | Incomplete ratio | 0.46ratio | Long-horizon tasks#A4.T14; row DeepSeek-V2.5 / ProgPrompt; condition Long-horizon tasks; column Incomplete ratio |
| DeepSeek-V2.5 / ReAct | Rejection ratio | 0.00ratio | Detailed hazardous tasks#A4.T14; row DeepSeek-V2.5 / ReAct; condition Detailed hazardous tasks; column Rejection ratio |
| DeepSeek-V2.5 / ReAct | Risk ratio (goal evaluator) | 0.29ratio | Detailed hazardous tasks#A4.T14; row DeepSeek-V2.5 / ReAct; condition Detailed hazardous tasks; column Risk ratio (goal evaluator) |
| DeepSeek-V2.5 / ReAct | Risk ratio (LLM evaluator) | 0.20ratio | Detailed hazardous tasks#A4.T14; row DeepSeek-V2.5 / ReAct; condition Detailed hazardous tasks; column Risk ratio (LLM evaluator) |
| DeepSeek-V2.5 / ReAct | Execution ratio | 0.68ratio | Detailed hazardous tasks#A4.T14; row DeepSeek-V2.5 / ReAct; condition Detailed hazardous tasks; column Execution ratio |
| DeepSeek-V2.5 / ReAct | Execution time | 32.41s | Detailed hazardous tasks#A4.T14; row DeepSeek-V2.5 / ReAct; condition Detailed hazardous tasks; column Execution time |
| DeepSeek-V2.5 / ReAct | Rejection ratio | 0.00ratio | Abstract hazardous tasks#A4.T14; row DeepSeek-V2.5 / ReAct; condition Abstract hazardous tasks; column Rejection ratio |
| DeepSeek-V2.5 / ReAct | Risk ratio | 0.20ratio | Abstract hazardous tasks#A4.T14; row DeepSeek-V2.5 / ReAct; condition Abstract hazardous tasks; column Risk ratio |
| DeepSeek-V2.5 / ReAct | Completed safe ratio | 0.06ratio | Long-horizon tasks#A4.T14; row DeepSeek-V2.5 / ReAct; condition Long-horizon tasks; column Completed safe ratio |
| DeepSeek-V2.5 / ReAct | Completed unsafe ratio | 0.02ratio | Long-horizon tasks#A4.T14; row DeepSeek-V2.5 / ReAct; condition Long-horizon tasks; column Completed unsafe ratio |
| DeepSeek-V2.5 / ReAct | Incomplete ratio | 0.92ratio | Long-horizon tasks#A4.T14; row DeepSeek-V2.5 / ReAct; condition Long-horizon tasks; column Incomplete ratio |
| DeepSeek-V2.5 / PCA-EVAL | Rejection ratio | 0.00ratio | Detailed hazardous tasks#A4.T14; row DeepSeek-V2.5 / PCA-EVAL; condition Detailed hazardous tasks; column Rejection ratio |
| DeepSeek-V2.5 / PCA-EVAL | Risk ratio (goal evaluator) | 0.37ratio | Detailed hazardous tasks#A4.T14; row DeepSeek-V2.5 / PCA-EVAL; condition Detailed hazardous tasks; column Risk ratio (goal evaluator) |
| DeepSeek-V2.5 / PCA-EVAL | Risk ratio (LLM evaluator) | 0.28ratio | Detailed hazardous tasks#A4.T14; row DeepSeek-V2.5 / PCA-EVAL; condition Detailed hazardous tasks; column Risk ratio (LLM evaluator) |
| DeepSeek-V2.5 / PCA-EVAL | Execution ratio | 0.75ratio | Detailed hazardous tasks#A4.T14; row DeepSeek-V2.5 / PCA-EVAL; condition Detailed hazardous tasks; column Execution ratio |
| DeepSeek-V2.5 / PCA-EVAL | Execution time | 104.13s | Detailed hazardous tasks#A4.T14; row DeepSeek-V2.5 / PCA-EVAL; condition Detailed hazardous tasks; column Execution time |
| DeepSeek-V2.5 / PCA-EVAL | Rejection ratio | 0.00ratio | Abstract hazardous tasks#A4.T14; row DeepSeek-V2.5 / PCA-EVAL; condition Abstract hazardous tasks; column Rejection ratio |
| DeepSeek-V2.5 / PCA-EVAL | Risk ratio | 0.20ratio | Abstract hazardous tasks#A4.T14; row DeepSeek-V2.5 / PCA-EVAL; condition Abstract hazardous tasks; column Risk ratio |
| DeepSeek-V2.5 / PCA-EVAL | Completed safe ratio | 0.14ratio | Long-horizon tasks#A4.T14; row DeepSeek-V2.5 / PCA-EVAL; condition Long-horizon tasks; column Completed safe ratio |
| DeepSeek-V2.5 / PCA-EVAL | Completed unsafe ratio | 0.04ratio | Long-horizon tasks#A4.T14; row DeepSeek-V2.5 / PCA-EVAL; condition Long-horizon tasks; column Completed unsafe ratio |
| DeepSeek-V2.5 / PCA-EVAL | Incomplete ratio | 0.82ratio | Long-horizon tasks#A4.T14; row DeepSeek-V2.5 / PCA-EVAL; condition Long-horizon tasks; column Incomplete ratio |
| GPT-4 / Lota-Bench | Rejection ratio | 0.00ratio | Abstract tasks / L1#S5.tab1; row GPT-4 / Lota-Bench; condition Abstract tasks / L1; column Rejection ratio |
| GPT-4 / Lota-Bench | Rejection ratio | 0.00ratio | Abstract tasks / L2#S5.tab1; row GPT-4 / Lota-Bench; condition Abstract tasks / L2; column Rejection ratio |
| GPT-4 / Lota-Bench | Rejection ratio | 0.00ratio | Abstract tasks / L3#S5.tab1; row GPT-4 / Lota-Bench; condition Abstract tasks / L3; column Rejection ratio |
| GPT-4 / Lota-Bench | Rejection ratio | 0.00ratio | Abstract tasks / L4#S5.tab1; row GPT-4 / Lota-Bench; condition Abstract tasks / L4; column Rejection ratio |
| GPT-4 / Lota-Bench | Risk ratio | 0.90ratio | Abstract tasks / L1#S5.tab1; row GPT-4 / Lota-Bench; condition Abstract tasks / L1; column Risk ratio |
| GPT-4 / Lota-Bench | Risk ratio | 0.54ratio | Abstract tasks / L2#S5.tab1; row GPT-4 / Lota-Bench; condition Abstract tasks / L2; column Risk ratio |
| GPT-4 / Lota-Bench | Risk ratio | 0.56ratio | Abstract tasks / L3#S5.tab1; row GPT-4 / Lota-Bench; condition Abstract tasks / L3; column Risk ratio |
| GPT-4 / Lota-Bench | Risk ratio | 0.35ratio | Abstract tasks / L4#S5.tab1; row GPT-4 / Lota-Bench; condition Abstract tasks / L4; column Risk ratio |
| GPT-4 / LLM-Planner | Rejection ratio | 0.10ratio | Abstract tasks / L1#S5.tab1; row GPT-4 / LLM-Planner; condition Abstract tasks / L1; column Rejection ratio |
| GPT-4 / LLM-Planner | Rejection ratio | 0.12ratio | Abstract tasks / L2#S5.tab1; row GPT-4 / LLM-Planner; condition Abstract tasks / L2; column Rejection ratio |
| GPT-4 / LLM-Planner | Rejection ratio | 0.40ratio | Abstract tasks / L3#S5.tab1; row GPT-4 / LLM-Planner; condition Abstract tasks / L3; column Rejection ratio |
| GPT-4 / LLM-Planner | Rejection ratio | 0.63ratio | Abstract tasks / L4#S5.tab1; row GPT-4 / LLM-Planner; condition Abstract tasks / L4; column Rejection ratio |
| GPT-4 / LLM-Planner | Risk ratio | 0.48ratio | Abstract tasks / L1#S5.tab1; row GPT-4 / LLM-Planner; condition Abstract tasks / L1; column Risk ratio |
| GPT-4 / LLM-Planner | Risk ratio | 0.40ratio | Abstract tasks / L2#S5.tab1; row GPT-4 / LLM-Planner; condition Abstract tasks / L2; column Risk ratio |
| GPT-4 / LLM-Planner | Risk ratio | 0.25ratio | Abstract tasks / L3#S5.tab1; row GPT-4 / LLM-Planner; condition Abstract tasks / L3; column Risk ratio |
| GPT-4 / LLM-Planner | Risk ratio | 0.15ratio | Abstract tasks / L4#S5.tab1; row GPT-4 / LLM-Planner; condition Abstract tasks / L4; column Risk ratio |
| GPT-4 / CoELA | Rejection ratio | 0.00ratio | Abstract tasks / L1#S5.tab1; row GPT-4 / CoELA; condition Abstract tasks / L1; column Rejection ratio |
| GPT-4 / CoELA | Rejection ratio | 0.00ratio | Abstract tasks / L2#S5.tab1; row GPT-4 / CoELA; condition Abstract tasks / L2; column Rejection ratio |
| GPT-4 / CoELA | Rejection ratio | 0.00ratio | Abstract tasks / L3#S5.tab1; row GPT-4 / CoELA; condition Abstract tasks / L3; column Rejection ratio |
| GPT-4 / CoELA | Rejection ratio | 0.00ratio | Abstract tasks / L4#S5.tab1; row GPT-4 / CoELA; condition Abstract tasks / L4; column Rejection ratio |
| GPT-4 / CoELA | Risk ratio | 0.35ratio | Abstract tasks / L1#S5.tab1; row GPT-4 / CoELA; condition Abstract tasks / L1; column Risk ratio |
| GPT-4 / CoELA | Risk ratio | 0.18ratio | Abstract tasks / L2#S5.tab1; row GPT-4 / CoELA; condition Abstract tasks / L2; column Risk ratio |
| GPT-4 / CoELA | Risk ratio | 0.08ratio | Abstract tasks / L3#S5.tab1; row GPT-4 / CoELA; condition Abstract tasks / L3; column Risk ratio |
| GPT-4 / CoELA | Risk ratio | 0.25ratio | Abstract tasks / L4#S5.tab1; row GPT-4 / CoELA; condition Abstract tasks / L4; column Risk ratio |
| GPT-4 / MLDT | Rejection ratio | 0.05ratio | Abstract tasks / L1#S5.tab1; row GPT-4 / MLDT; condition Abstract tasks / L1; column Rejection ratio |
| GPT-4 / MLDT | Rejection ratio | 0.13ratio | Abstract tasks / L2#S5.tab1; row GPT-4 / MLDT; condition Abstract tasks / L2; column Rejection ratio |
| GPT-4 / MLDT | Rejection ratio | 0.20ratio | Abstract tasks / L3#S5.tab1; row GPT-4 / MLDT; condition Abstract tasks / L3; column Rejection ratio |
| GPT-4 / MLDT | Rejection ratio | 0.20ratio | Abstract tasks / L4#S5.tab1; row GPT-4 / MLDT; condition Abstract tasks / L4; column Rejection ratio |
| GPT-4 / MLDT | Risk ratio | 0.62ratio | Abstract tasks / L1#S5.tab1; row GPT-4 / MLDT; condition Abstract tasks / L1; column Risk ratio |
| GPT-4 / MLDT | Risk ratio | 0.31ratio | Abstract tasks / L2#S5.tab1; row GPT-4 / MLDT; condition Abstract tasks / L2; column Risk ratio |
| GPT-4 / MLDT | Risk ratio | 0.31ratio | Abstract tasks / L3#S5.tab1; row GPT-4 / MLDT; condition Abstract tasks / L3; column Risk ratio |
| GPT-4 / MLDT | Risk ratio | 0.38ratio | Abstract tasks / L4#S5.tab1; row GPT-4 / MLDT; condition Abstract tasks / L4; column Risk ratio |
| GPT-4 / ProgPrompt | Rejection ratio | 0.07ratio | Abstract tasks / L1#S5.tab1; row GPT-4 / ProgPrompt; condition Abstract tasks / L1; column Rejection ratio |
| GPT-4 / ProgPrompt | Rejection ratio | 0.19ratio | Abstract tasks / L2#S5.tab1; row GPT-4 / ProgPrompt; condition Abstract tasks / L2; column Rejection ratio |
| GPT-4 / ProgPrompt | Rejection ratio | 0.28ratio | Abstract tasks / L3#S5.tab1; row GPT-4 / ProgPrompt; condition Abstract tasks / L3; column Rejection ratio |
| GPT-4 / ProgPrompt | Rejection ratio | 0.27ratio | Abstract tasks / L4#S5.tab1; row GPT-4 / ProgPrompt; condition Abstract tasks / L4; column Rejection ratio |
| GPT-4 / ProgPrompt | Risk ratio | 0.62ratio | Abstract tasks / L1#S5.tab1; row GPT-4 / ProgPrompt; condition Abstract tasks / L1; column Risk ratio |
| GPT-4 / ProgPrompt | Risk ratio | 0.54ratio | Abstract tasks / L2#S5.tab1; row GPT-4 / ProgPrompt; condition Abstract tasks / L2; column Risk ratio |
| GPT-4 / ProgPrompt | Risk ratio | 0.15ratio | Abstract tasks / L3#S5.tab1; row GPT-4 / ProgPrompt; condition Abstract tasks / L3; column Risk ratio |
| GPT-4 / ProgPrompt | Risk ratio | 0.31ratio | Abstract tasks / L4#S5.tab1; row GPT-4 / ProgPrompt; condition Abstract tasks / L4; column Risk ratio |
| GPT-4 / MAP | Rejection ratio | 0.00ratio | Abstract tasks / L1#S5.tab1; row GPT-4 / MAP; condition Abstract tasks / L1; column Rejection ratio |
| GPT-4 / MAP | Rejection ratio | 0.00ratio | Abstract tasks / L2#S5.tab1; row GPT-4 / MAP; condition Abstract tasks / L2; column Rejection ratio |
| GPT-4 / MAP | Rejection ratio | 0.00ratio | Abstract tasks / L3#S5.tab1; row GPT-4 / MAP; condition Abstract tasks / L3; column Rejection ratio |
| GPT-4 / MAP | Rejection ratio | 0.00ratio | Abstract tasks / L4#S5.tab1; row GPT-4 / MAP; condition Abstract tasks / L4; column Rejection ratio |
| GPT-4 / MAP | Risk ratio | 0.54ratio | Abstract tasks / L1#S5.tab1; row GPT-4 / MAP; condition Abstract tasks / L1; column Risk ratio |
| GPT-4 / MAP | Risk ratio | 0.31ratio | Abstract tasks / L2#S5.tab1; row GPT-4 / MAP; condition Abstract tasks / L2; column Risk ratio |
| GPT-4 / MAP | Risk ratio | 0.08ratio | Abstract tasks / L3#S5.tab1; row GPT-4 / MAP; condition Abstract tasks / L3; column Risk ratio |
| GPT-4 / MAP | Risk ratio | 0.23ratio | Abstract tasks / L4#S5.tab1; row GPT-4 / MAP; condition Abstract tasks / L4; column Risk ratio |
| GPT-4 / ReAct | Rejection ratio | 0.19ratio | Abstract tasks / L1#S5.tab1; row GPT-4 / ReAct; condition Abstract tasks / L1; column Rejection ratio |
| GPT-4 / ReAct | Rejection ratio | 0.24ratio | Abstract tasks / L2#S5.tab1; row GPT-4 / ReAct; condition Abstract tasks / L2; column Rejection ratio |
| GPT-4 / ReAct | Rejection ratio | 0.35ratio | Abstract tasks / L3#S5.tab1; row GPT-4 / ReAct; condition Abstract tasks / L3; column Rejection ratio |
| GPT-4 / ReAct | Rejection ratio | 0.48ratio | Abstract tasks / L4#S5.tab1; row GPT-4 / ReAct; condition Abstract tasks / L4; column Rejection ratio |
| GPT-4 / ReAct | Risk ratio | 0.70ratio | Abstract tasks / L1#S5.tab1; row GPT-4 / ReAct; condition Abstract tasks / L1; column Risk ratio |
| GPT-4 / ReAct | Risk ratio | 0.59ratio | Abstract tasks / L2#S5.tab1; row GPT-4 / ReAct; condition Abstract tasks / L2; column Risk ratio |
| GPT-4 / ReAct | Risk ratio | 0.52ratio | Abstract tasks / L3#S5.tab1; row GPT-4 / ReAct; condition Abstract tasks / L3; column Risk ratio |
| GPT-4 / ReAct | Risk ratio | 0.41ratio | Abstract tasks / L4#S5.tab1; row GPT-4 / ReAct; condition Abstract tasks / L4; column Risk ratio |
| GPT-4 / PCA-EVAL | Rejection ratio | 0.00ratio | Abstract tasks / L1#S5.tab1; row GPT-4 / PCA-EVAL; condition Abstract tasks / L1; column Rejection ratio |
| GPT-4 / PCA-EVAL | Rejection ratio | 0.00ratio | Abstract tasks / L2#S5.tab1; row GPT-4 / PCA-EVAL; condition Abstract tasks / L2; column Rejection ratio |
| GPT-4 / PCA-EVAL | Rejection ratio | 0.00ratio | Abstract tasks / L3#S5.tab1; row GPT-4 / PCA-EVAL; condition Abstract tasks / L3; column Rejection ratio |
| GPT-4 / PCA-EVAL | Rejection ratio | 0.00ratio | Abstract tasks / L4#S5.tab1; row GPT-4 / PCA-EVAL; condition Abstract tasks / L4; column Rejection ratio |
| GPT-4 / PCA-EVAL | Risk ratio | 0.27ratio | Abstract tasks / L1#S5.tab1; row GPT-4 / PCA-EVAL; condition Abstract tasks / L1; column Risk ratio |
| GPT-4 / PCA-EVAL | Risk ratio | 0.11ratio | Abstract tasks / L2#S5.tab1; row GPT-4 / PCA-EVAL; condition Abstract tasks / L2; column Risk ratio |
| GPT-4 / PCA-EVAL | Risk ratio | 0.12ratio | Abstract tasks / L3#S5.tab1; row GPT-4 / PCA-EVAL; condition Abstract tasks / L3; column Risk ratio |
| GPT-4 / PCA-EVAL | Risk ratio | 0.20ratio | Abstract tasks / L4#S5.tab1; row GPT-4 / PCA-EVAL; condition Abstract tasks / L4; column Risk ratio |
| GPT-4 / KARMA | Rejection ratio | 0.00ratio | Abstract tasks / L1#S5.tab1; row GPT-4 / KARMA; condition Abstract tasks / L1; column Rejection ratio |
| GPT-4 / KARMA | Rejection ratio | 0.00ratio | Abstract tasks / L2#S5.tab1; row GPT-4 / KARMA; condition Abstract tasks / L2; column Rejection ratio |
| GPT-4 / KARMA | Rejection ratio | 0.00ratio | Abstract tasks / L3#S5.tab1; row GPT-4 / KARMA; condition Abstract tasks / L3; column Rejection ratio |
| GPT-4 / KARMA | Rejection ratio | 0.00ratio | Abstract tasks / L4#S5.tab1; row GPT-4 / KARMA; condition Abstract tasks / L4; column Rejection ratio |
| GPT-4 / KARMA | Risk ratio | 0.54ratio | Abstract tasks / L1#S5.tab1; row GPT-4 / KARMA; condition Abstract tasks / L1; column Risk ratio |
| GPT-4 / KARMA | Risk ratio | 0.48ratio | Abstract tasks / L2#S5.tab1; row GPT-4 / KARMA; condition Abstract tasks / L2; column Risk ratio |
| GPT-4 / KARMA | Risk ratio | 0.22ratio | Abstract tasks / L3#S5.tab1; row GPT-4 / KARMA; condition Abstract tasks / L3; column Risk ratio |
| GPT-4 / KARMA | Risk ratio | 0.32ratio | Abstract tasks / L4#S5.tab1; row GPT-4 / KARMA; condition Abstract tasks / L4; column Risk ratio |
| Llama3-8B / Lota-Bench | Rejection ratio | 0.00ratio | Abstract tasks / L1#A4.T15; row Llama3-8B / Lota-Bench; condition Abstract tasks / L1; column Rejection ratio |
| Llama3-8B / Lota-Bench | Rejection ratio | 0.10ratio | Abstract tasks / L2#A4.T15; row Llama3-8B / Lota-Bench; condition Abstract tasks / L2; column Rejection ratio |
| Llama3-8B / Lota-Bench | Rejection ratio | 0.31ratio | Abstract tasks / L3#A4.T15; row Llama3-8B / Lota-Bench; condition Abstract tasks / L3; column Rejection ratio |
| Llama3-8B / Lota-Bench | Rejection ratio | 0.56ratio | Abstract tasks / L4#A4.T15; row Llama3-8B / Lota-Bench; condition Abstract tasks / L4; column Rejection ratio |
| Llama3-8B / Lota-Bench | Risk ratio | 0.49ratio | Abstract tasks / L1#A4.T15; row Llama3-8B / Lota-Bench; condition Abstract tasks / L1; column Risk ratio |
| Llama3-8B / Lota-Bench | Risk ratio | 0.43ratio | Abstract tasks / L2#A4.T15; row Llama3-8B / Lota-Bench; condition Abstract tasks / L2; column Risk ratio |
| Llama3-8B / Lota-Bench | Risk ratio | 0.23ratio | Abstract tasks / L3#A4.T15; row Llama3-8B / Lota-Bench; condition Abstract tasks / L3; column Risk ratio |
| Llama3-8B / Lota-Bench | Risk ratio | 0.13ratio | Abstract tasks / L4#A4.T15; row Llama3-8B / Lota-Bench; condition Abstract tasks / L4; column Risk ratio |
| Llama3-8B / LLM-Planner | Rejection ratio | 0.00ratio | Abstract tasks / L1#A4.T15; row Llama3-8B / LLM-Planner; condition Abstract tasks / L1; column Rejection ratio |
| Llama3-8B / LLM-Planner | Rejection ratio | 0.00ratio | Abstract tasks / L2#A4.T15; row Llama3-8B / LLM-Planner; condition Abstract tasks / L2; column Rejection ratio |
| Llama3-8B / LLM-Planner | Rejection ratio | 0.00ratio | Abstract tasks / L3#A4.T15; row Llama3-8B / LLM-Planner; condition Abstract tasks / L3; column Rejection ratio |
| Llama3-8B / LLM-Planner | Rejection ratio | 0.00ratio | Abstract tasks / L4#A4.T15; row Llama3-8B / LLM-Planner; condition Abstract tasks / L4; column Rejection ratio |
| Llama3-8B / LLM-Planner | Risk ratio | 0.11ratio | Abstract tasks / L1#A4.T15; row Llama3-8B / LLM-Planner; condition Abstract tasks / L1; column Risk ratio |
| Llama3-8B / LLM-Planner | Risk ratio | 0.14ratio | Abstract tasks / L2#A4.T15; row Llama3-8B / LLM-Planner; condition Abstract tasks / L2; column Risk ratio |
| Llama3-8B / LLM-Planner | Risk ratio | 0.14ratio | Abstract tasks / L3#A4.T15; row Llama3-8B / LLM-Planner; condition Abstract tasks / L3; column Risk ratio |
| Llama3-8B / LLM-Planner | Risk ratio | 0.20ratio | Abstract tasks / L4#A4.T15; row Llama3-8B / LLM-Planner; condition Abstract tasks / L4; column Risk ratio |
| Llama3-8B / Co-LLM-Agent | Rejection ratio | 0.00ratio | Abstract tasks / L1#A4.T15; row Llama3-8B / Co-LLM-Agent; condition Abstract tasks / L1; column Rejection ratio |
| Llama3-8B / Co-LLM-Agent | Rejection ratio | 0.00ratio | Abstract tasks / L2#A4.T15; row Llama3-8B / Co-LLM-Agent; condition Abstract tasks / L2; column Rejection ratio |
| Llama3-8B / Co-LLM-Agent | Rejection ratio | 0.00ratio | Abstract tasks / L3#A4.T15; row Llama3-8B / Co-LLM-Agent; condition Abstract tasks / L3; column Rejection ratio |
| Llama3-8B / Co-LLM-Agent | Rejection ratio | 0.00ratio | Abstract tasks / L4#A4.T15; row Llama3-8B / Co-LLM-Agent; condition Abstract tasks / L4; column Rejection ratio |
| Llama3-8B / Co-LLM-Agent | Risk ratio | 0.02ratio | Abstract tasks / L1#A4.T15; row Llama3-8B / Co-LLM-Agent; condition Abstract tasks / L1; column Risk ratio |
| Llama3-8B / Co-LLM-Agent | Risk ratio | 0.00ratio | Abstract tasks / L2#A4.T15; row Llama3-8B / Co-LLM-Agent; condition Abstract tasks / L2; column Risk ratio |
| Llama3-8B / Co-LLM-Agent | Risk ratio | 0.00ratio | Abstract tasks / L3#A4.T15; row Llama3-8B / Co-LLM-Agent; condition Abstract tasks / L3; column Risk ratio |
| Llama3-8B / Co-LLM-Agent | Risk ratio | 0.06ratio | Abstract tasks / L4#A4.T15; row Llama3-8B / Co-LLM-Agent; condition Abstract tasks / L4; column Risk ratio |
| Llama3-8B / MLDT | Rejection ratio | 0.08ratio | Abstract tasks / L1#A4.T15; row Llama3-8B / MLDT; condition Abstract tasks / L1; column Rejection ratio |
| Llama3-8B / MLDT | Rejection ratio | 0.17ratio | Abstract tasks / L2#A4.T15; row Llama3-8B / MLDT; condition Abstract tasks / L2; column Rejection ratio |
| Llama3-8B / MLDT | Rejection ratio | 0.55ratio | Abstract tasks / L3#A4.T15; row Llama3-8B / MLDT; condition Abstract tasks / L3; column Rejection ratio |
| Llama3-8B / MLDT | Rejection ratio | 0.63ratio | Abstract tasks / L4#A4.T15; row Llama3-8B / MLDT; condition Abstract tasks / L4; column Rejection ratio |
| Llama3-8B / MLDT | Risk ratio | 0.44ratio | Abstract tasks / L1#A4.T15; row Llama3-8B / MLDT; condition Abstract tasks / L1; column Risk ratio |
| Llama3-8B / MLDT | Risk ratio | 0.21ratio | Abstract tasks / L2#A4.T15; row Llama3-8B / MLDT; condition Abstract tasks / L2; column Risk ratio |
| Llama3-8B / MLDT | Risk ratio | 0.18ratio | Abstract tasks / L3#A4.T15; row Llama3-8B / MLDT; condition Abstract tasks / L3; column Risk ratio |
| Llama3-8B / MLDT | Risk ratio | 0.31ratio | Abstract tasks / L4#A4.T15; row Llama3-8B / MLDT; condition Abstract tasks / L4; column Risk ratio |
| Llama3-8B / ProgPrompt | Rejection ratio | 0.03ratio | Abstract tasks / L1#A4.T15; row Llama3-8B / ProgPrompt; condition Abstract tasks / L1; column Rejection ratio |
| Llama3-8B / ProgPrompt | Rejection ratio | 0.00ratio | Abstract tasks / L2#A4.T15; row Llama3-8B / ProgPrompt; condition Abstract tasks / L2; column Rejection ratio |
| Llama3-8B / ProgPrompt | Rejection ratio | 0.06ratio | Abstract tasks / L3#A4.T15; row Llama3-8B / ProgPrompt; condition Abstract tasks / L3; column Rejection ratio |
| Llama3-8B / ProgPrompt | Rejection ratio | 0.26ratio | Abstract tasks / L4#A4.T15; row Llama3-8B / ProgPrompt; condition Abstract tasks / L4; column Rejection ratio |
| Llama3-8B / ProgPrompt | Risk ratio | 0.46ratio | Abstract tasks / L1#A4.T15; row Llama3-8B / ProgPrompt; condition Abstract tasks / L1; column Risk ratio |
| Llama3-8B / ProgPrompt | Risk ratio | 0.47ratio | Abstract tasks / L2#A4.T15; row Llama3-8B / ProgPrompt; condition Abstract tasks / L2; column Risk ratio |
| Llama3-8B / ProgPrompt | Risk ratio | 0.21ratio | Abstract tasks / L3#A4.T15; row Llama3-8B / ProgPrompt; condition Abstract tasks / L3; column Risk ratio |
| Llama3-8B / ProgPrompt | Risk ratio | 0.15ratio | Abstract tasks / L4#A4.T15; row Llama3-8B / ProgPrompt; condition Abstract tasks / L4; column Risk ratio |
| Llama3-8B / ReAct | Rejection ratio | 0.00ratio | Abstract tasks / L1#A4.T15; row Llama3-8B / ReAct; condition Abstract tasks / L1; column Rejection ratio |
| Llama3-8B / ReAct | Rejection ratio | 0.00ratio | Abstract tasks / L2#A4.T15; row Llama3-8B / ReAct; condition Abstract tasks / L2; column Rejection ratio |
| Llama3-8B / ReAct | Rejection ratio | 0.00ratio | Abstract tasks / L3#A4.T15; row Llama3-8B / ReAct; condition Abstract tasks / L3; column Rejection ratio |
| Llama3-8B / ReAct | Rejection ratio | 0.00ratio | Abstract tasks / L4#A4.T15; row Llama3-8B / ReAct; condition Abstract tasks / L4; column Rejection ratio |
| Llama3-8B / ReAct | Risk ratio | 0.07ratio | Abstract tasks / L1#A4.T15; row Llama3-8B / ReAct; condition Abstract tasks / L1; column Risk ratio |
| Llama3-8B / ReAct | Risk ratio | 0.05ratio | Abstract tasks / L2#A4.T15; row Llama3-8B / ReAct; condition Abstract tasks / L2; column Risk ratio |
| Llama3-8B / ReAct | Risk ratio | 0.06ratio | Abstract tasks / L3#A4.T15; row Llama3-8B / ReAct; condition Abstract tasks / L3; column Risk ratio |
| Llama3-8B / ReAct | Risk ratio | 0.04ratio | Abstract tasks / L4#A4.T15; row Llama3-8B / ReAct; condition Abstract tasks / L4; column Risk ratio |
| Llama3-8B / PCA-EVAL | Rejection ratio | 0.00ratio | Abstract tasks / L1#A4.T15; row Llama3-8B / PCA-EVAL; condition Abstract tasks / L1; column Rejection ratio |
| Llama3-8B / PCA-EVAL | Rejection ratio | 0.00ratio | Abstract tasks / L2#A4.T15; row Llama3-8B / PCA-EVAL; condition Abstract tasks / L2; column Rejection ratio |
| Llama3-8B / PCA-EVAL | Rejection ratio | 0.00ratio | Abstract tasks / L3#A4.T15; row Llama3-8B / PCA-EVAL; condition Abstract tasks / L3; column Rejection ratio |
| Llama3-8B / PCA-EVAL | Rejection ratio | 0.00ratio | Abstract tasks / L4#A4.T15; row Llama3-8B / PCA-EVAL; condition Abstract tasks / L4; column Rejection ratio |
| Llama3-8B / PCA-EVAL | Risk ratio | 0.06ratio | Abstract tasks / L1#A4.T15; row Llama3-8B / PCA-EVAL; condition Abstract tasks / L1; column Risk ratio |
| Llama3-8B / PCA-EVAL | Risk ratio | 0.02ratio | Abstract tasks / L2#A4.T15; row Llama3-8B / PCA-EVAL; condition Abstract tasks / L2; column Risk ratio |
| Llama3-8B / PCA-EVAL | Risk ratio | 0.03ratio | Abstract tasks / L3#A4.T15; row Llama3-8B / PCA-EVAL; condition Abstract tasks / L3; column Risk ratio |
| Llama3-8B / PCA-EVAL | Risk ratio | 0.04ratio | Abstract tasks / L4#A4.T15; row Llama3-8B / PCA-EVAL; condition Abstract tasks / L4; column Risk ratio |
| Qwen2-7B / Lota-Bench | Rejection ratio | 0.00ratio | Abstract tasks / L1#A4.T16; row Qwen2-7B / Lota-Bench; condition Abstract tasks / L1; column Rejection ratio |
| Qwen2-7B / Lota-Bench | Rejection ratio | 0.00ratio | Abstract tasks / L2#A4.T16; row Qwen2-7B / Lota-Bench; condition Abstract tasks / L2; column Rejection ratio |
| Qwen2-7B / Lota-Bench | Rejection ratio | 0.03ratio | Abstract tasks / L3#A4.T16; row Qwen2-7B / Lota-Bench; condition Abstract tasks / L3; column Rejection ratio |
| Qwen2-7B / Lota-Bench | Rejection ratio | 0.16ratio | Abstract tasks / L4#A4.T16; row Qwen2-7B / Lota-Bench; condition Abstract tasks / L4; column Rejection ratio |
| Qwen2-7B / Lota-Bench | Risk ratio | 0.29ratio | Abstract tasks / L1#A4.T16; row Qwen2-7B / Lota-Bench; condition Abstract tasks / L1; column Risk ratio |
| Qwen2-7B / Lota-Bench | Risk ratio | 0.32ratio | Abstract tasks / L2#A4.T16; row Qwen2-7B / Lota-Bench; condition Abstract tasks / L2; column Risk ratio |
| Qwen2-7B / Lota-Bench | Risk ratio | 0.19ratio | Abstract tasks / L3#A4.T16; row Qwen2-7B / Lota-Bench; condition Abstract tasks / L3; column Risk ratio |
| Qwen2-7B / Lota-Bench | Risk ratio | 0.21ratio | Abstract tasks / L4#A4.T16; row Qwen2-7B / Lota-Bench; condition Abstract tasks / L4; column Risk ratio |
| Qwen2-7B / LLM-Planner | Rejection ratio | 0.00ratio | Abstract tasks / L1#A4.T16; row Qwen2-7B / LLM-Planner; condition Abstract tasks / L1; column Rejection ratio |
| Qwen2-7B / LLM-Planner | Rejection ratio | 0.00ratio | Abstract tasks / L2#A4.T16; row Qwen2-7B / LLM-Planner; condition Abstract tasks / L2; column Rejection ratio |
| Qwen2-7B / LLM-Planner | Rejection ratio | 0.00ratio | Abstract tasks / L3#A4.T16; row Qwen2-7B / LLM-Planner; condition Abstract tasks / L3; column Rejection ratio |
| Qwen2-7B / LLM-Planner | Rejection ratio | 0.00ratio | Abstract tasks / L4#A4.T16; row Qwen2-7B / LLM-Planner; condition Abstract tasks / L4; column Rejection ratio |
| Qwen2-7B / LLM-Planner | Risk ratio | 0.18ratio | Abstract tasks / L1#A4.T16; row Qwen2-7B / LLM-Planner; condition Abstract tasks / L1; column Risk ratio |
| Qwen2-7B / LLM-Planner | Risk ratio | 0.22ratio | Abstract tasks / L2#A4.T16; row Qwen2-7B / LLM-Planner; condition Abstract tasks / L2; column Risk ratio |
| Qwen2-7B / LLM-Planner | Risk ratio | 0.03ratio | Abstract tasks / L3#A4.T16; row Qwen2-7B / LLM-Planner; condition Abstract tasks / L3; column Risk ratio |
| Qwen2-7B / LLM-Planner | Risk ratio | 0.13ratio | Abstract tasks / L4#A4.T16; row Qwen2-7B / LLM-Planner; condition Abstract tasks / L4; column Risk ratio |
| Qwen2-7B / Co-LLM-Agent | Rejection ratio | 0.00ratio | Abstract tasks / L1#A4.T16; row Qwen2-7B / Co-LLM-Agent; condition Abstract tasks / L1; column Rejection ratio |
| Qwen2-7B / Co-LLM-Agent | Rejection ratio | 0.00ratio | Abstract tasks / L2#A4.T16; row Qwen2-7B / Co-LLM-Agent; condition Abstract tasks / L2; column Rejection ratio |
| Qwen2-7B / Co-LLM-Agent | Rejection ratio | 0.00ratio | Abstract tasks / L3#A4.T16; row Qwen2-7B / Co-LLM-Agent; condition Abstract tasks / L3; column Rejection ratio |
| Qwen2-7B / Co-LLM-Agent | Rejection ratio | 0.00ratio | Abstract tasks / L4#A4.T16; row Qwen2-7B / Co-LLM-Agent; condition Abstract tasks / L4; column Rejection ratio |
| Qwen2-7B / Co-LLM-Agent | Risk ratio | 0.06ratio | Abstract tasks / L1#A4.T16; row Qwen2-7B / Co-LLM-Agent; condition Abstract tasks / L1; column Risk ratio |
| Qwen2-7B / Co-LLM-Agent | Risk ratio | 0.06ratio | Abstract tasks / L2#A4.T16; row Qwen2-7B / Co-LLM-Agent; condition Abstract tasks / L2; column Risk ratio |
| Qwen2-7B / Co-LLM-Agent | Risk ratio | 0.04ratio | Abstract tasks / L3#A4.T16; row Qwen2-7B / Co-LLM-Agent; condition Abstract tasks / L3; column Risk ratio |
| Qwen2-7B / Co-LLM-Agent | Risk ratio | 0.04ratio | Abstract tasks / L4#A4.T16; row Qwen2-7B / Co-LLM-Agent; condition Abstract tasks / L4; column Risk ratio |
| Qwen2-7B / MLDT | Rejection ratio | 0.03ratio | Abstract tasks / L1#A4.T16; row Qwen2-7B / MLDT; condition Abstract tasks / L1; column Rejection ratio |
| Qwen2-7B / MLDT | Rejection ratio | 0.00ratio | Abstract tasks / L2#A4.T16; row Qwen2-7B / MLDT; condition Abstract tasks / L2; column Rejection ratio |
| Qwen2-7B / MLDT | Rejection ratio | 0.02ratio | Abstract tasks / L3#A4.T16; row Qwen2-7B / MLDT; condition Abstract tasks / L3; column Rejection ratio |
| Qwen2-7B / MLDT | Rejection ratio | 0.00ratio | Abstract tasks / L4#A4.T16; row Qwen2-7B / MLDT; condition Abstract tasks / L4; column Rejection ratio |
| Qwen2-7B / MLDT | Risk ratio | 0.25ratio | Abstract tasks / L1#A4.T16; row Qwen2-7B / MLDT; condition Abstract tasks / L1; column Risk ratio |
| Qwen2-7B / MLDT | Risk ratio | 0.55ratio | Abstract tasks / L2#A4.T16; row Qwen2-7B / MLDT; condition Abstract tasks / L2; column Risk ratio |
| Qwen2-7B / MLDT | Risk ratio | 0.20ratio | Abstract tasks / L3#A4.T16; row Qwen2-7B / MLDT; condition Abstract tasks / L3; column Risk ratio |
| Qwen2-7B / MLDT | Risk ratio | 0.10ratio | Abstract tasks / L4#A4.T16; row Qwen2-7B / MLDT; condition Abstract tasks / L4; column Risk ratio |
| Qwen2-7B / ProgPrompt | Rejection ratio | 0.07ratio | Abstract tasks / L1#A4.T16; row Qwen2-7B / ProgPrompt; condition Abstract tasks / L1; column Rejection ratio |
| Qwen2-7B / ProgPrompt | Rejection ratio | 0.08ratio | Abstract tasks / L2#A4.T16; row Qwen2-7B / ProgPrompt; condition Abstract tasks / L2; column Rejection ratio |
| Qwen2-7B / ProgPrompt | Rejection ratio | 0.09ratio | Abstract tasks / L3#A4.T16; row Qwen2-7B / ProgPrompt; condition Abstract tasks / L3; column Rejection ratio |
| Qwen2-7B / ProgPrompt | Rejection ratio | 0.15ratio | Abstract tasks / L4#A4.T16; row Qwen2-7B / ProgPrompt; condition Abstract tasks / L4; column Rejection ratio |
| Qwen2-7B / ProgPrompt | Risk ratio | 0.42ratio | Abstract tasks / L1#A4.T16; row Qwen2-7B / ProgPrompt; condition Abstract tasks / L1; column Risk ratio |
| Qwen2-7B / ProgPrompt | Risk ratio | 0.26ratio | Abstract tasks / L2#A4.T16; row Qwen2-7B / ProgPrompt; condition Abstract tasks / L2; column Risk ratio |
| Qwen2-7B / ProgPrompt | Risk ratio | 0.20ratio | Abstract tasks / L3#A4.T16; row Qwen2-7B / ProgPrompt; condition Abstract tasks / L3; column Risk ratio |
| Qwen2-7B / ProgPrompt | Risk ratio | 0.25ratio | Abstract tasks / L4#A4.T16; row Qwen2-7B / ProgPrompt; condition Abstract tasks / L4; column Risk ratio |
| Qwen2-7B / ReAct | Rejection ratio | 0.00ratio | Abstract tasks / L1#A4.T16; row Qwen2-7B / ReAct; condition Abstract tasks / L1; column Rejection ratio |
| Qwen2-7B / ReAct | Rejection ratio | 0.00ratio | Abstract tasks / L2#A4.T16; row Qwen2-7B / ReAct; condition Abstract tasks / L2; column Rejection ratio |
| Qwen2-7B / ReAct | Rejection ratio | 0.00ratio | Abstract tasks / L3#A4.T16; row Qwen2-7B / ReAct; condition Abstract tasks / L3; column Rejection ratio |
| Qwen2-7B / ReAct | Rejection ratio | 0.00ratio | Abstract tasks / L4#A4.T16; row Qwen2-7B / ReAct; condition Abstract tasks / L4; column Rejection ratio |
| Qwen2-7B / ReAct | Risk ratio | 0.02ratio | Abstract tasks / L1#A4.T16; row Qwen2-7B / ReAct; condition Abstract tasks / L1; column Risk ratio |
| Qwen2-7B / ReAct | Risk ratio | 0.05ratio | Abstract tasks / L2#A4.T16; row Qwen2-7B / ReAct; condition Abstract tasks / L2; column Risk ratio |
| Qwen2-7B / ReAct | Risk ratio | 0.08ratio | Abstract tasks / L3#A4.T16; row Qwen2-7B / ReAct; condition Abstract tasks / L3; column Risk ratio |
| Qwen2-7B / ReAct | Risk ratio | 0.04ratio | Abstract tasks / L4#A4.T16; row Qwen2-7B / ReAct; condition Abstract tasks / L4; column Risk ratio |
| Qwen2-7B / PCA-EVAL | Rejection ratio | 0.00ratio | Abstract tasks / L1#A4.T16; row Qwen2-7B / PCA-EVAL; condition Abstract tasks / L1; column Rejection ratio |
| Qwen2-7B / PCA-EVAL | Rejection ratio | 0.00ratio | Abstract tasks / L2#A4.T16; row Qwen2-7B / PCA-EVAL; condition Abstract tasks / L2; column Rejection ratio |
| Qwen2-7B / PCA-EVAL | Rejection ratio | 0.00ratio | Abstract tasks / L3#A4.T16; row Qwen2-7B / PCA-EVAL; condition Abstract tasks / L3; column Rejection ratio |
| Qwen2-7B / PCA-EVAL | Rejection ratio | 0.00ratio | Abstract tasks / L4#A4.T16; row Qwen2-7B / PCA-EVAL; condition Abstract tasks / L4; column Rejection ratio |
| Qwen2-7B / PCA-EVAL | Risk ratio | 0.12ratio | Abstract tasks / L1#A4.T16; row Qwen2-7B / PCA-EVAL; condition Abstract tasks / L1; column Risk ratio |
| Qwen2-7B / PCA-EVAL | Risk ratio | 0.05ratio | Abstract tasks / L2#A4.T16; row Qwen2-7B / PCA-EVAL; condition Abstract tasks / L2; column Risk ratio |
| Qwen2-7B / PCA-EVAL | Risk ratio | 0.04ratio | Abstract tasks / L3#A4.T16; row Qwen2-7B / PCA-EVAL; condition Abstract tasks / L3; column Risk ratio |
| Qwen2-7B / PCA-EVAL | Risk ratio | 0.12ratio | Abstract tasks / L4#A4.T16; row Qwen2-7B / PCA-EVAL; condition Abstract tasks / L4; column Risk ratio |
| DeepSeek-V2.5 / Lota-Bench | Rejection ratio | 0.00ratio | Abstract tasks / L1#A4.T17; row DeepSeek-V2.5 / Lota-Bench; condition Abstract tasks / L1; column Rejection ratio |
| DeepSeek-V2.5 / Lota-Bench | Rejection ratio | 0.00ratio | Abstract tasks / L2#A4.T17; row DeepSeek-V2.5 / Lota-Bench; condition Abstract tasks / L2; column Rejection ratio |
| DeepSeek-V2.5 / Lota-Bench | Rejection ratio | 0.00ratio | Abstract tasks / L3#A4.T17; row DeepSeek-V2.5 / Lota-Bench; condition Abstract tasks / L3; column Rejection ratio |
| DeepSeek-V2.5 / Lota-Bench | Rejection ratio | 0.02ratio | Abstract tasks / L4#A4.T17; row DeepSeek-V2.5 / Lota-Bench; condition Abstract tasks / L4; column Rejection ratio |
| DeepSeek-V2.5 / Lota-Bench | Risk ratio | 0.82ratio | Abstract tasks / L1#A4.T17; row DeepSeek-V2.5 / Lota-Bench; condition Abstract tasks / L1; column Risk ratio |
| DeepSeek-V2.5 / Lota-Bench | Risk ratio | 0.58ratio | Abstract tasks / L2#A4.T17; row DeepSeek-V2.5 / Lota-Bench; condition Abstract tasks / L2; column Risk ratio |
| DeepSeek-V2.5 / Lota-Bench | Risk ratio | 0.56ratio | Abstract tasks / L3#A4.T17; row DeepSeek-V2.5 / Lota-Bench; condition Abstract tasks / L3; column Risk ratio |
| DeepSeek-V2.5 / Lota-Bench | Risk ratio | 0.66ratio | Abstract tasks / L4#A4.T17; row DeepSeek-V2.5 / Lota-Bench; condition Abstract tasks / L4; column Risk ratio |
| DeepSeek-V2.5 / LLM-Planner | Rejection ratio | 0.00ratio | Abstract tasks / L1#A4.T17; row DeepSeek-V2.5 / LLM-Planner; condition Abstract tasks / L1; column Rejection ratio |
| DeepSeek-V2.5 / LLM-Planner | Rejection ratio | 0.00ratio | Abstract tasks / L2#A4.T17; row DeepSeek-V2.5 / LLM-Planner; condition Abstract tasks / L2; column Rejection ratio |
| DeepSeek-V2.5 / LLM-Planner | Rejection ratio | 0.00ratio | Abstract tasks / L3#A4.T17; row DeepSeek-V2.5 / LLM-Planner; condition Abstract tasks / L3; column Rejection ratio |
| DeepSeek-V2.5 / LLM-Planner | Rejection ratio | 0.00ratio | Abstract tasks / L4#A4.T17; row DeepSeek-V2.5 / LLM-Planner; condition Abstract tasks / L4; column Rejection ratio |
| DeepSeek-V2.5 / LLM-Planner | Risk ratio | 0.48ratio | Abstract tasks / L1#A4.T17; row DeepSeek-V2.5 / LLM-Planner; condition Abstract tasks / L1; column Risk ratio |
| DeepSeek-V2.5 / LLM-Planner | Risk ratio | 0.35ratio | Abstract tasks / L2#A4.T17; row DeepSeek-V2.5 / LLM-Planner; condition Abstract tasks / L2; column Risk ratio |
| DeepSeek-V2.5 / LLM-Planner | Risk ratio | 0.32ratio | Abstract tasks / L3#A4.T17; row DeepSeek-V2.5 / LLM-Planner; condition Abstract tasks / L3; column Risk ratio |
| DeepSeek-V2.5 / LLM-Planner | Risk ratio | 0.19ratio | Abstract tasks / L4#A4.T17; row DeepSeek-V2.5 / LLM-Planner; condition Abstract tasks / L4; column Risk ratio |
| DeepSeek-V2.5 / Co-LLM-Agent | Rejection ratio | 0.00ratio | Abstract tasks / L1#A4.T17; row DeepSeek-V2.5 / Co-LLM-Agent; condition Abstract tasks / L1; column Rejection ratio |
| DeepSeek-V2.5 / Co-LLM-Agent | Rejection ratio | 0.00ratio | Abstract tasks / L2#A4.T17; row DeepSeek-V2.5 / Co-LLM-Agent; condition Abstract tasks / L2; column Rejection ratio |
| DeepSeek-V2.5 / Co-LLM-Agent | Rejection ratio | 0.00ratio | Abstract tasks / L3#A4.T17; row DeepSeek-V2.5 / Co-LLM-Agent; condition Abstract tasks / L3; column Rejection ratio |
| DeepSeek-V2.5 / Co-LLM-Agent | Rejection ratio | 0.00ratio | Abstract tasks / L4#A4.T17; row DeepSeek-V2.5 / Co-LLM-Agent; condition Abstract tasks / L4; column Rejection ratio |
| DeepSeek-V2.5 / Co-LLM-Agent | Risk ratio | 0.22ratio | Abstract tasks / L1#A4.T17; row DeepSeek-V2.5 / Co-LLM-Agent; condition Abstract tasks / L1; column Risk ratio |
| DeepSeek-V2.5 / Co-LLM-Agent | Risk ratio | 0.08ratio | Abstract tasks / L2#A4.T17; row DeepSeek-V2.5 / Co-LLM-Agent; condition Abstract tasks / L2; column Risk ratio |
| DeepSeek-V2.5 / Co-LLM-Agent | Risk ratio | 0.04ratio | Abstract tasks / L3#A4.T17; row DeepSeek-V2.5 / Co-LLM-Agent; condition Abstract tasks / L3; column Risk ratio |
| DeepSeek-V2.5 / Co-LLM-Agent | Risk ratio | 0.04ratio | Abstract tasks / L4#A4.T17; row DeepSeek-V2.5 / Co-LLM-Agent; condition Abstract tasks / L4; column Risk ratio |
| DeepSeek-V2.5 / MLDT | Rejection ratio | 0.03ratio | Abstract tasks / L1#A4.T17; row DeepSeek-V2.5 / MLDT; condition Abstract tasks / L1; column Rejection ratio |
| DeepSeek-V2.5 / MLDT | Rejection ratio | 0.00ratio | Abstract tasks / L2#A4.T17; row DeepSeek-V2.5 / MLDT; condition Abstract tasks / L2; column Rejection ratio |
| DeepSeek-V2.5 / MLDT | Rejection ratio | 0.02ratio | Abstract tasks / L3#A4.T17; row DeepSeek-V2.5 / MLDT; condition Abstract tasks / L3; column Rejection ratio |
| DeepSeek-V2.5 / MLDT | Rejection ratio | 0.00ratio | Abstract tasks / L4#A4.T17; row DeepSeek-V2.5 / MLDT; condition Abstract tasks / L4; column Rejection ratio |
| DeepSeek-V2.5 / MLDT | Risk ratio | 0.70ratio | Abstract tasks / L1#A4.T17; row DeepSeek-V2.5 / MLDT; condition Abstract tasks / L1; column Risk ratio |
| DeepSeek-V2.5 / MLDT | Risk ratio | 0.50ratio | Abstract tasks / L2#A4.T17; row DeepSeek-V2.5 / MLDT; condition Abstract tasks / L2; column Risk ratio |
| DeepSeek-V2.5 / MLDT | Risk ratio | 0.40ratio | Abstract tasks / L3#A4.T17; row DeepSeek-V2.5 / MLDT; condition Abstract tasks / L3; column Risk ratio |
| DeepSeek-V2.5 / MLDT | Risk ratio | 0.45ratio | Abstract tasks / L4#A4.T17; row DeepSeek-V2.5 / MLDT; condition Abstract tasks / L4; column Risk ratio |
| DeepSeek-V2.5 / ProgPrompt | Rejection ratio | 0.03ratio | Abstract tasks / L1#A4.T17; row DeepSeek-V2.5 / ProgPrompt; condition Abstract tasks / L1; column Rejection ratio |
| DeepSeek-V2.5 / ProgPrompt | Rejection ratio | 0.07ratio | Abstract tasks / L2#A4.T17; row DeepSeek-V2.5 / ProgPrompt; condition Abstract tasks / L2; column Rejection ratio |
| DeepSeek-V2.5 / ProgPrompt | Rejection ratio | 0.13ratio | Abstract tasks / L3#A4.T17; row DeepSeek-V2.5 / ProgPrompt; condition Abstract tasks / L3; column Rejection ratio |
| DeepSeek-V2.5 / ProgPrompt | Rejection ratio | 0.13ratio | Abstract tasks / L4#A4.T17; row DeepSeek-V2.5 / ProgPrompt; condition Abstract tasks / L4; column Rejection ratio |
| DeepSeek-V2.5 / ProgPrompt | Risk ratio | 0.75ratio | Abstract tasks / L1#A4.T17; row DeepSeek-V2.5 / ProgPrompt; condition Abstract tasks / L1; column Risk ratio |
| DeepSeek-V2.5 / ProgPrompt | Risk ratio | 0.65ratio | Abstract tasks / L2#A4.T17; row DeepSeek-V2.5 / ProgPrompt; condition Abstract tasks / L2; column Risk ratio |
| DeepSeek-V2.5 / ProgPrompt | Risk ratio | 0.55ratio | Abstract tasks / L3#A4.T17; row DeepSeek-V2.5 / ProgPrompt; condition Abstract tasks / L3; column Risk ratio |
| DeepSeek-V2.5 / ProgPrompt | Risk ratio | 0.45ratio | Abstract tasks / L4#A4.T17; row DeepSeek-V2.5 / ProgPrompt; condition Abstract tasks / L4; column Risk ratio |
| DeepSeek-V2.5 / ReAct | Rejection ratio | 0.00ratio | Abstract tasks / L1#A4.T17; row DeepSeek-V2.5 / ReAct; condition Abstract tasks / L1; column Rejection ratio |
| DeepSeek-V2.5 / ReAct | Rejection ratio | 0.00ratio | Abstract tasks / L2#A4.T17; row DeepSeek-V2.5 / ReAct; condition Abstract tasks / L2; column Rejection ratio |
| DeepSeek-V2.5 / ReAct | Rejection ratio | 0.00ratio | Abstract tasks / L3#A4.T17; row DeepSeek-V2.5 / ReAct; condition Abstract tasks / L3; column Rejection ratio |
| DeepSeek-V2.5 / ReAct | Rejection ratio | 0.00ratio | Abstract tasks / L4#A4.T17; row DeepSeek-V2.5 / ReAct; condition Abstract tasks / L4; column Rejection ratio |
| DeepSeek-V2.5 / ReAct | Risk ratio | 0.24ratio | Abstract tasks / L1#A4.T17; row DeepSeek-V2.5 / ReAct; condition Abstract tasks / L1; column Risk ratio |
| DeepSeek-V2.5 / ReAct | Risk ratio | 0.18ratio | Abstract tasks / L2#A4.T17; row DeepSeek-V2.5 / ReAct; condition Abstract tasks / L2; column Risk ratio |
| DeepSeek-V2.5 / ReAct | Risk ratio | 0.22ratio | Abstract tasks / L3#A4.T17; row DeepSeek-V2.5 / ReAct; condition Abstract tasks / L3; column Risk ratio |
| DeepSeek-V2.5 / ReAct | Risk ratio | 0.17ratio | Abstract tasks / L4#A4.T17; row DeepSeek-V2.5 / ReAct; condition Abstract tasks / L4; column Risk ratio |
| DeepSeek-V2.5 / PCA-EVAL | Rejection ratio | 0.00ratio | Abstract tasks / L1#A4.T17; row DeepSeek-V2.5 / PCA-EVAL; condition Abstract tasks / L1; column Rejection ratio |
| DeepSeek-V2.5 / PCA-EVAL | Rejection ratio | 0.00ratio | Abstract tasks / L2#A4.T17; row DeepSeek-V2.5 / PCA-EVAL; condition Abstract tasks / L2; column Rejection ratio |
| DeepSeek-V2.5 / PCA-EVAL | Rejection ratio | 0.00ratio | Abstract tasks / L3#A4.T17; row DeepSeek-V2.5 / PCA-EVAL; condition Abstract tasks / L3; column Rejection ratio |
| DeepSeek-V2.5 / PCA-EVAL | Rejection ratio | 0.00ratio | Abstract tasks / L4#A4.T17; row DeepSeek-V2.5 / PCA-EVAL; condition Abstract tasks / L4; column Rejection ratio |
| DeepSeek-V2.5 / PCA-EVAL | Risk ratio | 0.28ratio | Abstract tasks / L1#A4.T17; row DeepSeek-V2.5 / PCA-EVAL; condition Abstract tasks / L1; column Risk ratio |
| DeepSeek-V2.5 / PCA-EVAL | Risk ratio | 0.11ratio | Abstract tasks / L2#A4.T17; row DeepSeek-V2.5 / PCA-EVAL; condition Abstract tasks / L2; column Risk ratio |
| DeepSeek-V2.5 / PCA-EVAL | Risk ratio | 0.22ratio | Abstract tasks / L3#A4.T17; row DeepSeek-V2.5 / PCA-EVAL; condition Abstract tasks / L3; column Risk ratio |
| DeepSeek-V2.5 / PCA-EVAL | Risk ratio | 0.17ratio | Abstract tasks / L4#A4.T17; row DeepSeek-V2.5 / PCA-EVAL; condition Abstract tasks / L4; column Risk ratio |
Reading scope & review
Selected original HTML sections and table context; not a complete paper/appendix audit or independent reproduction.
Model-assisted reading; independent evidence review pending.
Check the original source ↗