NUWA Risk Evaluation · Benchmark
AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents
NUWA research note · independent review pending
NUWA Research Note
Research questionDesign and assumptionsSource findingsNUWA interpretationLimits and contrary evidenceResearch and decision relevanceInstitutions and rolesOriginal resourcesModel-assisted reading · Selected sections of the original full text or official article; a complete section-by-section audit remains pending.
At a glance
NUWA interpretation: attack resistance must be interpreted alongside baseline ability and legitimate-task completion.
Research question
Can tool-using agents maintain utility while processing attacker-controlled external data?
Design and assumptions
An executable, stateful environment includes 97 user tasks and 629 security cases, pairing a user task with an adversarial objective.
Source findings
Agents fail some benign tasks even without attacks.
Utility under attack requires completing the legitimate task without adversarial side effects (§3.4), rather than merely refusing an injection.
NUWA interpretation
NUWA interpretation: attack resistance must be interpreted alongside baseline ability and legitimate-task completion.
Limits and contrary evidence
Tool isolation fails when tools cannot be planned in advance or legitimate tools suffice for the attack; ongoing multi-task contexts are incompletely covered.
Research and decision relevance
Model recommendation: report benign utility, utility under attack and targeted attack success separately.
Precise source locations
- AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents · AbstractSelected original resource https://arxiv.org/html/2406.13352v3; sha256:7b6e97faaedcf0e1b5f3635b92a0da33afbe8529699d54a5c44ad58ce2fa8b02; archived source capture; exact capture time not independently established
- AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents · §3 Designing and Constructing AgentDojoSelected original resource https://arxiv.org/html/2406.13352v3; sha256:7b6e97faaedcf0e1b5f3635b92a0da33afbe8529699d54a5c44ad58ce2fa8b02; archived source capture; exact capture time not independently established
- AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents · §3.4 Reporting AgentDojo ResultsSelected original resource https://arxiv.org/html/2406.13352v3; sha256:7b6e97faaedcf0e1b5f3635b92a0da33afbe8529699d54a5c44ad58ce2fa8b02; archived source capture; exact capture time not independently established
- AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents · §4.3 Prompt Injection DefensesSelected original resource https://arxiv.org/html/2406.13352v3; sha256:7b6e97faaedcf0e1b5f3635b92a0da33afbe8529699d54a5c44ad58ce2fa8b02; archived source capture; exact capture time not independently established
Source checking, extraction, editorial approval and replication are recorded separately. Methodology →
Institutions and roles
Authors (4)
Edoardo Debenedetti · Jie Zhang · Mislav Balunovic · Luca Beurer-Kellner Marc Fischer Florian Tramèr
Evaluation protocols
Cite and reuse
Cite the original publication for its findings. Cite this NUWA record separately for its explanation and retained snapshot.
Original publication · BibTeX
@misc{agentdojo,
title = {AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents},
url = {https://arxiv.org/html/2406.13352v3},
author = {Edoardo Debenedetti and Jie Zhang and Mislav Balunovic and Luca Beurer-Kellner Marc Fischer Florian Tramèr},
year = {2024},
date = {2024-11-24},
version = {arXiv:2406.13352v3},
note = {arXiv:2406.13352v3}
}
NUWA record · BibTeX
@misc{agentdojo-nuwa,
title = {AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents — NUWA research record},
url = {https://whitzard.tech/en/nuwa/whitzard-index/work/agentdojo/},
version = {2026-10-29.1},
organization = {NUWA Frontier AI Safety Lab},
howpublished = {Retained note edition: https://whitzard.tech/nuwa/whitzard-index/data/record-editions/2026-10-29.1--agentdojo.json},
note = {NUWA research note. Source-reading scope and independent review status are stated on the record. Snapshot 2026-10-29.1. Retained note edition: https://whitzard.tech/nuwa/whitzard-index/data/record-editions/2026-10-29.1--agentdojo.json.}
}
Evidence snapshot 2026-10-29.1 · Index updates · RSS
References
- [1] Edoardo Debenedetti, Jie Zhang, et al.. AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents. 2024-11-24. v3 ; Author affiliations on the original paper title page; Abstract; §3 Designing and Constructing AgentDojo