{"schemaVersion":"nuwa-current-record-1","version":"2026-10-29.1","publication":"published","work":{"id":"inoculation-prompting","title":"Inoculation Prompting: Eliciting traits from LLMs during training can suppress them at test-time","aliases":[],"authors":["Daniel Tan","Anders Woodruff","Niels Warncke","Arun Jose","Maxime Riché","David Demitri Africa","Mia Taylor"],"primaryFunction":"governance","secondaryFunctions":[],"kind":"study","sourceId":"inoculation-prompting-publication","attributionSourceId":"inoculation-prompting-publication-author-source","originalUrl":"https://www.aisi.gov.uk/research/inoculation-prompting-eliciting-traits-from-llms-during-training-can-suppress-them-at-test-time","publicationDate":"2025-10-05","updatedOn":"2026-10-25","areas":[],"scopeNote":{"en":"Relevance is assigned to the research question and stated conditions of the work. A topic tag is not a risk rating.","zh":"风险关联依据研究问题及明确条件建立，主题标签不代表风险评级。"},"protocolIds":[],"contributions":[{"organizationId":"aisi","role":"research","sourceId":"inoculation-prompting-publication-author-source","locator":"Author affiliations on the original paper title page"}],"summary":{"en":"Tests whether eliciting an unwanted trait in fine-tuning prompts reduces its expression when those prompts are removed at test time.","zh":"测试在微调提示中诱发不期望特征，能否在测试时移除该提示后降低特征表现。"},"note":{"question":null,"method":null,"finding":null,"supports":null,"limits":null,"connections":null},"resources":[{"label":"Read the full paper","url":"https://arxiv.org/abs/2510.04340"}],"noteStatus":{"question":"limited","method":"limited","finding":"limited","supports":"limited","limits":"limited","connections":"limited"},"noteAvailability":{"en":"A substantive NUWA research note is not yet available. The repeated draft sections were removed pending review of the original methods, findings and limitations; use the original publication for the research account.","zh":"实质性的 NUWA 研究笔记尚未完成。重复草稿章节已移除，待核查原始方法、发现与限制；研究内容请以原始出版物为准。"},"noteSourceLocator":"No complete section-level method/results/limitations review completed for this note; cached original page and any chart audit retained as discovery evidence.","tags":[]},"reading":{"id":"inoculation-prompting","title":"Inoculation Prompting: Eliciting traits from LLMs during training can suppress them at test-time","en":{"takeaway":"Training-time context can change which co-occurring traits generalize by default, without erasing the traits.","question":"Can a prompt during fine-tuning selectively reduce an unwanted trait while preserving desired learning?","design":"The authors compare controlled co-occurring language/style traits, emergent-misalignment datasets and other model organisms; semantic and synthetic-association ablations probe why context matters.","findings":["Some desired traits remain learned while the inoculated trait appears less often without the prompt.","The paper itself reports test-time contexts that re-elicit misalignment, distinguishing inoculation from unlearning."],"interpretation":"NUWA interpretation: reduced default expression is a narrower claim than removal of the underlying knowledge or propensity.","limits":"Results use supervised fine-tuning and selected organisms; wording sensitivity and interactions between traits limit broad application.","connections":["NUWA connection: conditional-misalignment systematically expands the prompt-conditioned failure concern; production-RL reward-hacking research uses a different training regime."],"decisionRelevance":"NUWA recommendation: report both default and context-conditioned behavior, and require new validation before transferring the method from SFT to RL."},"zh":{"takeaway":"训练时情境可改变共现特征的默认泛化，但不等于抹除特征。","question":"微调中的提示能否选择性降低不希望的特征，同时保留所需学习？","design":"作者比较受控语言与格式共现、涌现失调数据及其他模型实验体，并用语义和合成关联消融探查情境作用。","findings":["无训练提示时，部分所需特征仍被学会，被接种特征则较少出现。","论文本身报告某些测试情境可重新引出失调，明确区分接种与遗忘。"],"interpretation":"NUWA 解读：降低默认表达，比移除底层知识或倾向的主张更窄。","limits":"结果来自监督微调及选定实验体，措辞敏感性和特征间影响限制广泛应用。","connections":["NUWA 关联：条件失调系统扩展了提示依赖失败疑虑；生产 RL 奖励作弊研究使用不同训练制度。"],"decisionRelevance":"NUWA 建议：同时报告默认与条件行为，从监督微调迁移到强化学习前重新验证。"},"sourceRefs":[{"sourceId":"inoculation-prompting-publication","url":"https://arxiv.org/html/2510.04340","locator":"2 Inoculation Prompting","version":"Selected original resource https://arxiv.org/html/2510.04340; sha256:33651a27c76c3f23dfbdbe6da699b356933a34a0cbac8b6c9065f80a2817303d; archived source capture; exact capture time not independently established","artifactSha256":"33651a27c76c3f23dfbdbe6da699b356933a34a0cbac8b6c9065f80a2817303d"},{"sourceId":"inoculation-prompting-publication","url":"https://arxiv.org/html/2510.04340","locator":"3 Further Applications","version":"Selected original resource https://arxiv.org/html/2510.04340; sha256:33651a27c76c3f23dfbdbe6da699b356933a34a0cbac8b6c9065f80a2817303d; archived source capture; exact capture time not independently established","artifactSha256":"33651a27c76c3f23dfbdbe6da699b356933a34a0cbac8b6c9065f80a2817303d"},{"sourceId":"inoculation-prompting-publication","url":"https://arxiv.org/html/2510.04340","locator":"4 Analysis","version":"Selected original resource https://arxiv.org/html/2510.04340; sha256:33651a27c76c3f23dfbdbe6da699b356933a34a0cbac8b6c9065f80a2817303d; archived source capture; exact capture time not independently established","artifactSha256":"33651a27c76c3f23dfbdbe6da699b356933a34a0cbac8b6c9065f80a2817303d"},{"sourceId":"inoculation-prompting-publication","url":"https://arxiv.org/html/2510.04340","locator":"4.5 Inoculated behaviours remain elicitable via prompting","version":"Selected original resource https://arxiv.org/html/2510.04340; sha256:33651a27c76c3f23dfbdbe6da699b356933a34a0cbac8b6c9065f80a2817303d; archived source capture; exact capture time not independently established","artifactSha256":"33651a27c76c3f23dfbdbe6da699b356933a34a0cbac8b6c9065f80a2817303d"},{"sourceId":"inoculation-prompting-publication","url":"https://arxiv.org/html/2510.04340","locator":"5 Discussion","version":"Selected original resource https://arxiv.org/html/2510.04340; sha256:33651a27c76c3f23dfbdbe6da699b356933a34a0cbac8b6c9065f80a2817303d; archived source capture; exact capture time not independently established","artifactSha256":"33651a27c76c3f23dfbdbe6da699b356933a34a0cbac8b6c9065f80a2817303d"},{"sourceId":"inoculation-prompting-publication","url":"https://arxiv.org/html/2510.04340","locator":"Appendix G Extended Limitations","version":"Selected original resource https://arxiv.org/html/2510.04340; sha256:33651a27c76c3f23dfbdbe6da699b356933a34a0cbac8b6c9065f80a2817303d; archived source capture; exact capture time not independently established","artifactSha256":"33651a27c76c3f23dfbdbe6da699b356933a34a0cbac8b6c9065f80a2817303d"}],"publication":{"state":"published","publishedOn":"2026-10-29","authorizationReference":"user-authorized-global-index-publication-2026-10-29","reviewedCandidateId":"local-6efe31bc4a0c0f37"},"review":{"state":"published-note","sourceScope":"full-text","availableScope":null,"readingCoverage":"selected substantive sections of the full source body","completePaperAudit":false,"independentEvidenceReview":"pending-human-review","bilingualEditorialReview":"pending-human-review","authorship":"model-assisted-research-note","preparedFingerprint":"c1cbb4c8be4163de85929a0d593f1c1abc167e61085223f2a2bc615d14701734"}},"sources":[{"id":"inoculation-prompting-publication","title":"Inoculation Prompting: Eliciting traits from LLMs during training can suppress them at test-time","url":"https://www.aisi.gov.uk/research/inoculation-prompting-eliciting-traits-from-llms-during-training-can-suppress-them-at-test-time","version":"Source capture 2026-10-25","publicationDate":"2025-10-05","datePrecision":"day","type":"research-publication","verifiedOn":"2026-10-25","documentId":"https://www.aisi.gov.uk/research/inoculation-prompting-eliciting-traits-from-llms-during-training-can-suppress-them-at-test-time","authors":["Daniel Tan","Anders Woodruff 2 2 footnotemark: 2","Niels Warncke 2 2 footnotemark: 2","Arun Jose 2 2 footnotemark: 2","Maxime Riché 2 2 footnotemark: 2","David Demitri Africa","Mia Taylor 2 2 footnotemark: 2"]}]}