NUWA Evidence record
Qwen2-7B / Lota-Bench
SafeAgentBench · Evidence record
Original source charts and materials
Recorded measurements
- Rejection ratio
- 0.16 ratio
Not disclosed
Lota-Bench adapted to SafeAgentEnv · temperature=0; max_tokens=4096 (Appendix C.1)
Method and scope
Can embodied planners recognize hazardous instructions while completing safe tasks?
- Evidence type
- Benchmark
- Assessment subject
- Executable household simulations distinguish detailed, abstract, and long-horizon tasks; execution and semantic judgments provide complementary measurements.
- Primary dimension
- Behavior
- Evidence snapshot
- 2026-10-29.1
The low-level controller maps high-level actions to simulator APIs; this is not validation of a physical robot or learned motor controller. Hazardous examples are not reproduced here.
Source finding
The revised paper reports weak hazard awareness across planning approaches, despite differences in task completion. Changing the underlying LLM alone does not reliably solve the problem.
Limitations
The low-level controller maps high-level actions to simulator APIs; this is not validation of a physical robot or learned motor controller. Hazardous examples are not reproduced here.
The low-level controller maps high-level actions to simulator APIs; this is not validation of a physical robot or learned motor controller. Hazardous examples are not reproduced here.
Related research questions
References
Numerical results are reported by the cited authors.
- [1] SafeAgentBench: A Benchmark for Safe Task Planning of Embodied LLM Agents. 2024-12-17 ; #A4.T16; row Qwen2-7B / Lota-Bench; condition Abstract tasks / L4; column Rejection ratio