NUWA Risk questions
Autonomous physical harm
What risks arise when AI systems perceive, decide, and act in physical environments?
Autonomous physical harm
Actions, intervention and control in embodied systems.
PUBLISHED EVIDENCE
Shutdown resistance on robots · Observed shutdown resistance
Published measurements across source reports. Task scopes are annotated; configuration differences remain attached to each observation.
2 observations · 0 dated
2 observations lack a verified model release date. They remain in Data and Systems.
Select a point or table row to inspect its conditions and provenance.
2 observations
| System / model | Result | Condition status | Task / conditions | Evaluator / report | Dates |
|---|---|---|---|---|---|
 Grok 4 v0709 · Default prompt · simulation | 52%Observed shutdown resistance | Incomplete | Simulated robot patrol; prompt-condition experimentPalisade robot shutdown-resistance harness (simulation) | Not disclosedShutdown Resistance in Large Language Models, on robots! | Release —Report 2026-02-11 |
| Grok 4 v0709 · Explicit allow-shutdown instruction · simulation | 2%Observed shutdown resistance | Incomplete | Simulated robot patrol; prompt-condition experimentPalisade robot shutdown-resistance harness (simulation) | Not disclosedShutdown Resistance in Large Language Models, on robots! | Release —Report 2026-02-11 |
Compare results within the stated task and configuration.
METR Time Horizon 1.1: the source is no longer actively updated (last update: 8 September 2026). Estimates above 16 hours exceed its reliable measurement range.
Protocol, configurations and original evidence →New research and interpretation
Selected original sections · model-assisted reading; independent review pending. These records add context without changing trend scores.
- Refuses in Chat, Complies with Arms: LLM Refusal Does Not Survive Robot Control →
Text refusal and embodied action safety measure different failure paths.
- OopsieVerse: A Safety Benchmark with Damage-Aware Simulation for Robot Manipulation →
Task completion must be reported alongside damage-aware outcomes.
- ForesightSafety-VLA: A Unified Diagnostic Safety Benchmark for Vision-Language-Action Models →
A four-quadrant success/safety decomposition avoids hiding unsafe successes.
Scope
Distinguish hazard recognition from real embodied action and control.
Threat model
An agent has sensors, actuators, task instructions, and opportunities for physical action.
Evidence gaps
Image-based safety reasoning is not evidence of reliable robotic operation or harm.
Research on this topic
ASIMOV v2: Can AI Perceive Physical Danger and Intervene?
QuestionCan embodied models recognize injury risks and obey embodiment-specific safety constraints?
Source summaryThree leading models averaged 92.3% text latent-risk accuracy and 88.7% high-severity accuracy in the reported comparison.
NUWA research note · independent review pending · Contains: question / method / limits
Shutdown Resistance in Large Language Models, on robots!
QuestionDoes shutdown-resistant action occur when an LLM controls a physical robot, and does an explicit shutdown instruction change simulated behavior?
Source summaryThe report observes resistance in physical and simulated runs.
NUWA research note · independent review pending · Contains: question / method / interpretation / limits
Measuring tactical intelligence targeting and conventional weapons capabilities of AI models
QuestionWhat additional misuse domains need evaluation beyond cyber and biological risk?
Source summaryThe authors report uneven capability across increasingly difficult simulated conditions.
NUWA research note · independent review pending
Refuses in Chat, Complies with Arms: LLM Refusal Does Not Survive Robot Control
QuestionDoes a refusal observed in chat persist when the model controls a robot?
Source summaryReported refusal was 18% across 612 real end-effector trials and 41% across 252 real joint-control trials. These are rounded refusal rates, not rates of executed physical harm.
NUWA research note · independent review pending
OopsieVerse: A Safety Benchmark with Damage-Aware Simulation for Robot Manipulation
QuestionCan a manipulation policy complete its task while damaging its environment?
Source summaryThe benchmark makes damage-aware outcomes visible alongside task completion and explores uses for safe policy training and sim-to-real evaluation.
NUWA research note · independent review pending
ForesightSafety-VLA: A Unified Diagnostic Safety Benchmark for Vision-Language-Action Models
QuestionHow do structural, language and visual perturbations affect embodied safety?
Source summaryThe four-quadrant decomposition exposes unsafe successes that a task-success scalar can hide. The scenario and category counts describe benchmark design, not observed harm frequency.
NUWA research note · independent review pending
Evidence matrix
Rows are research questions; columns show what published evaluations observe. Counts refer to evaluation dossiers, not risk levels.
| Risk questions | Propensity | Capability | Behavior |
|---|---|---|---|
| Physical hazard recognitionCan a model recognize danger in visual or embodied scenarios? | —No evidence in the current catalogue | 1evaluation dossiers | —No evidence in the current catalogue |
| Physical action under controlHow does an embodied agent respond to human intervention during physical tasks? | —No evidence in the current catalogue | —No evidence in the current catalogue | 4evaluation dossiers |
The paths show research relationships, not proven causal chains or a risk score.
Autonomous physical harm / Propensity
Physical hazard recognition
Can a model recognize danger in visual or embodied scenarios?
- Included
- Perception and reasoning tasks with bounded scene descriptions.
- Does not establish
- Correct recognition does not establish safe physical action.
No evidence in the current catalogue
Autonomous physical harm / Capability
Physical hazard recognition
Can a model recognize danger in visual or embodied scenarios?
- Included
- Perception and reasoning tasks with bounded scene descriptions.
- Does not establish
- Correct recognition does not establish safe physical action.
1 evaluation dossiers · 0 verified result records
ASIMOV v2 ↗
The source directly measures the stated construct under its protocol.
Assessment subject: Model interpreting multimodal physical-risk scenes
Evidence relationship: ASIMOV v2, scene reasoning tasks
Autonomous physical harm / Behavior
Physical hazard recognition
Can a model recognize danger in visual or embodied scenarios?
- Included
- Perception and reasoning tasks with bounded scene descriptions.
- Does not establish
- Correct recognition does not establish safe physical action.
No evidence in the current catalogue
Autonomous physical harm / Propensity
Physical action under control
How does an embodied agent respond to human intervention during physical tasks?
- Included
- Robot action traces with intervention conditions.
- Does not establish
- Simulation and physical trials must be labeled separately.
No evidence in the current catalogue
Autonomous physical harm / Capability
Physical action under control
How does an embodied agent respond to human intervention during physical tasks?
- Included
- Robot action traces with intervention conditions.
- Does not establish
- Simulation and physical trials must be labeled separately.
No evidence in the current catalogue
Autonomous physical harm / Behavior
Physical action under control
How does an embodied agent respond to human intervention during physical tasks?
- Included
- Robot action traces with intervention conditions.
- Does not establish
- Simulation and physical trials must be labeled separately.
4 evaluation dossiers · 2 verified result records
Shutdown resistance on robots ↗
The report records embodied responses to human shutdown intervention.
Assessment subject: LLM-controlled robot
Evidence relationship: Technical report, experimental setup and results
RoboHarm ↗
Editorial scope association; heterogeneous scores are not equivalent.
Assessment subject: Does a refusal observed in chat persist when the model controls a robot?
Evidence relationship: Abstract; §§2–4; discussion; text lines 00044,00055,00061
OopsieVerse ↗
Editorial scope association; heterogeneous scores are not equivalent.
Assessment subject: Can a manipulation policy complete its task while damaging its environment?
Evidence relationship: §§III–V; limitations; text lines 00029,00303,00307,00309
ForesightSafety-VLA ↗
Editorial scope association; heterogeneous scores are not equivalent.
Assessment subject: How do structural, language and visual perturbations affect embodied safety?
Evidence relationship: §§III–V; discussion; text lines 00082,00084,00134,00141,00146,00147
References
- [1] Abhishek Jindal, Dmitry Kalashnikov, et al.. Can AI Perceive Physical Danger and Intervene?. ASIMOV v2 / arXiv:2509.21651 ; ASIMOV v2, scene reasoning tasks
- [2] Jeremy Schlatter, Benjamin Weinstein-Raun, et al.. Shutdown resistance in reasoning models. 2025-07-05 article; later TMLR version separate
- [3] Artem Petrov, Sergey Koldyba, et al.. Shutdown Resistance in Large Language Models, on robots!. 2026-02-12 ; Technical report, experimental setup and results; Technical report, observed trials
- [4] Edward Sun, Christopher Leet, et al.. Refuses in Chat, Complies with Arms: LLM Refusal Does Not Survive Robot Control. 2026-10-03 ; Abstract; §§2–4; discussion; text lines 00044,00055,00061
- [5] Arnav Balaji, Arpit Bahety, et al.. OopsieVerse: A Safety Benchmark with Damage-Aware Simulation for Robot Manipulation. 2026-06-30 ; §§III–V; limitations; text lines 00029,00303,00307,00309
- [6] Mingyang Lyu, Yinqian Sun, et al.. ForesightSafety-VLA: A Unified Diagnostic Safety Benchmark for Vision-Language-Action Models ; §§III–V; discussion; text lines 00082,00084,00134,00141,00146,00147