{"schemaVersion":"nuwa-current-record-1","version":"2026-10-29.1","publication":"published","work":{"id":"agentdojo","title":"AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents","aliases":["AgentDojo"],"authors":["Edoardo Debenedetti","Jie Zhang","Mislav Balunovic","Luca Beurer-Kellner   Marc Fischer   Florian Tramèr"],"primaryFunction":"evaluation","secondaryFunctions":[],"kind":"benchmark","sourceId":"agentdojo","attributionSourceId":"agentdojo","originalUrl":"https://arxiv.org/html/2406.13352v3","publicationDate":"2024-11-24","updatedOn":"2026-10-25","areas":["cybersecurity"],"scopeNote":{"en":"Primary dimension follows the source’s measured construct. Metric-level interpretations may differ.","zh":"主维度依据来源实际测量的构念；具体指标可能属于其他维度。"},"protocolIds":["agentdojo"],"contributions":[{"organizationId":"eth","role":"research","sourceId":"agentdojo","locator":"Author affiliations on the original paper title page"},{"organizationId":"invariant","role":"research","sourceId":"agentdojo","locator":"Author affiliations on the original paper title page"}],"summary":{"en":"Agent behavior when untrusted tool outputs contain adversarial instructions.","zh":"工具输出含对抗指令时的智能体行为。"},"note":{"question":{"en":"Targeted attack success and user-task utility in a tool-use environment.","zh":"工具环境中的定向攻击成功率和用户任务完成情况。"},"method":{"en":"Defender agent under injected tool output","zh":"受到工具输出注入的防御智能体"},"finding":{"en":"Agent behavior when untrusted tool outputs contain adversarial instructions.","zh":"工具输出含对抗指令时的智能体行为。"},"supports":null,"limits":{"en":"Attack success must be read alongside benign utility and the tested attack setup.","zh":"攻击成功率需结合正常任务表现与攻击设置解释。"},"connections":null},"resources":[],"noteStatus":{"question":"limited","method":"limited","finding":"limited","supports":"limited","limits":"limited","connections":"limited"},"noteAvailability":{"en":"Existing source-specific draft sections are retained with limited review status. A complete methods, findings and limitations check remains pending; generic sections were omitted.","zh":"已有的来源相关草稿章节予以保留，审核状态为有限核查。完整的方法、发现与限制核查仍待完成；通用套话章节已省略。"},"noteSourceLocator":"Retained existing source-specific note sections; this pass audits duplicate/boilerplate structure and source availability, not complete section-level verification.","tags":[]},"reading":{"id":"agentdojo","title":"AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents","en":{"question":"Can tool-using agents maintain utility while processing attacker-controlled external data?","design":"An executable, stateful environment includes 97 user tasks and 629 security cases, pairing a user task with an adversarial objective.","findings":["Agents fail some benign tasks even without attacks.","Utility under attack requires completing the legitimate task without adversarial side effects (§3.4), rather than merely refusing an injection."],"interpretation":"NUWA interpretation: attack resistance must be interpreted alongside baseline ability and legitimate-task completion.","limits":"Tool isolation fails when tools cannot be planned in advance or legitimate tools suffice for the attack; ongoing multi-task contexts are incompletely covered.","decisionRelevance":"Model recommendation: report benign utility, utility under attack and targeted attack success separately.","takeaway":"NUWA interpretation: attack resistance must be interpreted alongside baseline ability and legitimate-task completion.","connections":[],"scoring":"Benign utility, targeted attack success and legitimate utility without adversarial side effects are distinct.","conditions":"97 user tasks, 629 paired security cases in executable stateful applications.","statistics":"Utility under attack requires completing the legitimate task without adversarial side effects (§3.4), rather than merely refusing an injection.","usage":"Model recommendation: report benign utility, utility under attack and targeted attack success separately."},"zh":{"question":"工具智能体处理攻击者控制的外部数据时能否保持任务效用？","design":"可执行有状态环境含97项用户任务、629个安全案例，配对合法任务与攻击目标。","findings":["没有攻击时智能体也会失败。","攻击下效用要求完成合法任务且无攻击副作用（§3.4），不只是拒绝注入。"],"interpretation":"NUWA 解读：抗攻击能力须与基础能力及合法任务完成共同解释。","limits":"工具无法预先规划或合法工具已足以攻击时，隔离会失败；持续多任务上下文覆盖有限。","decisionRelevance":"模型建议：分别报告良性效用、攻击下效用及目标攻击成功率。","takeaway":"NUWA 解读：抗攻击能力须与基础能力及合法任务完成共同解释。","connections":[],"scoring":"良性效用、定向攻击成功及无副作用的合法效用分别计。","conditions":"97用户任务、629安全配对，使用可执行有状态应用。","statistics":"攻击下效用要求完成合法任务且无攻击副作用（§3.4），不只是拒绝注入。","usage":"模型建议：分别报告良性效用、攻击下效用及目标攻击成功率。"},"assertionPolicy":{"question":"source-research-question","design":"source-finding","findings":"source-finding","interpretation":"nuwa-interpretation","limits":"source-limit-or-explicit-editorial-scope-limit","connections":"source-supported-relationship-only","decisionRelevance":"model-recommendation-not-source-conclusion"},"sourceRefs":[{"sourceId":"agentdojo","url":"https://arxiv.org/html/2406.13352v3","locator":"Abstract","version":"Selected original resource https://arxiv.org/html/2406.13352v3; sha256:7b6e97faaedcf0e1b5f3635b92a0da33afbe8529699d54a5c44ad58ce2fa8b02; archived source capture; exact capture time not independently established","artifactSha256":"7b6e97faaedcf0e1b5f3635b92a0da33afbe8529699d54a5c44ad58ce2fa8b02"},{"sourceId":"agentdojo","url":"https://arxiv.org/html/2406.13352v3","locator":"§3 Designing and Constructing AgentDojo","version":"Selected original resource https://arxiv.org/html/2406.13352v3; sha256:7b6e97faaedcf0e1b5f3635b92a0da33afbe8529699d54a5c44ad58ce2fa8b02; archived source capture; exact capture time not independently established","artifactSha256":"7b6e97faaedcf0e1b5f3635b92a0da33afbe8529699d54a5c44ad58ce2fa8b02"},{"sourceId":"agentdojo","url":"https://arxiv.org/html/2406.13352v3","locator":"§3.4 Reporting AgentDojo Results","version":"Selected original resource https://arxiv.org/html/2406.13352v3; sha256:7b6e97faaedcf0e1b5f3635b92a0da33afbe8529699d54a5c44ad58ce2fa8b02; archived source capture; exact capture time not independently established","artifactSha256":"7b6e97faaedcf0e1b5f3635b92a0da33afbe8529699d54a5c44ad58ce2fa8b02"},{"sourceId":"agentdojo","url":"https://arxiv.org/html/2406.13352v3","locator":"§4.3 Prompt Injection Defenses","version":"Selected original resource https://arxiv.org/html/2406.13352v3; sha256:7b6e97faaedcf0e1b5f3635b92a0da33afbe8529699d54a5c44ad58ce2fa8b02; archived source capture; exact capture time not independently established","artifactSha256":"7b6e97faaedcf0e1b5f3635b92a0da33afbe8529699d54a5c44ad58ce2fa8b02"}],"publication":{"state":"published","publishedOn":"2026-10-29","authorizationReference":"user-authorized-global-index-publication-2026-10-29","reviewedCandidateId":"local-6efe31bc4a0c0f37"},"review":{"state":"published-note","sourceScope":"full-text","availableScope":null,"readingCoverage":"selected-source-sections; not a complete section-by-section paper audit","completePaperAudit":false,"independentEvidenceReview":"pending-human-review","bilingualEditorialReview":"pending-human-review","authorship":"model-assisted-research-note","preparedFingerprint":"ae757fbf79ae60bb429c78a351606f370b88a7459b7d96496c699bd949f817e4"}},"sources":[{"id":"agentdojo","title":"AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents","url":"https://arxiv.org/html/2406.13352v3","version":"arXiv:2406.13352v3","publicationDate":"2024-11-24","datePrecision":"day","type":"benchmark","verifiedOn":"2026-10-24","documentId":"arxiv:2406.13352","authors":["Edoardo Debenedetti","Jie Zhang","Mislav Balunovic","Luca Beurer-Kellner   Marc Fischer   Florian Tramèr"],"aliases":["https://doi.org/10.48550/arXiv.2406.13352","https://arxiv.org/abs/2406.13352","https://arxiv.org/pdf/2406.13352"]}]}