NUWA Risk Governance · Study
Inoculation Prompting: Eliciting traits from LLMs during training can suppress them at test-time
NUWA research note · independent review pending
NUWA Research Note
Research questionDesign and assumptionsSource findingsNUWA interpretationLimits and contrary evidenceResearch connectionsResearch and decision relevanceInstitutions and rolesOriginal resourcesModel-assisted reading · Selected sections of the original full text or official article; a complete section-by-section audit remains pending.
At a glance
Training-time context can change which co-occurring traits generalize by default, without erasing the traits.
Research question
Can a prompt during fine-tuning selectively reduce an unwanted trait while preserving desired learning?
Design and assumptions
The authors compare controlled co-occurring language/style traits, emergent-misalignment datasets and other model organisms; semantic and synthetic-association ablations probe why context matters.
Source findings
Some desired traits remain learned while the inoculated trait appears less often without the prompt.
The paper itself reports test-time contexts that re-elicit misalignment, distinguishing inoculation from unlearning.
NUWA interpretation
NUWA interpretation: reduced default expression is a narrower claim than removal of the underlying knowledge or propensity.
Limits and contrary evidence
Results use supervised fine-tuning and selected organisms; wording sensitivity and interactions between traits limit broad application.
Research connections
NUWA connection: conditional-misalignment systematically expands the prompt-conditioned failure concern; production-RL reward-hacking research uses a different training regime.
Research and decision relevance
NUWA recommendation: report both default and context-conditioned behavior, and require new validation before transferring the method from SFT to RL.
Precise source locations
- Inoculation Prompting: Eliciting traits from LLMs during training can suppress them at test-time · 2 Inoculation PromptingSelected original resource https://arxiv.org/html/2510.04340; sha256:33651a27c76c3f23dfbdbe6da699b356933a34a0cbac8b6c9065f80a2817303d; archived source capture; exact capture time not independently established
- Inoculation Prompting: Eliciting traits from LLMs during training can suppress them at test-time · 3 Further ApplicationsSelected original resource https://arxiv.org/html/2510.04340; sha256:33651a27c76c3f23dfbdbe6da699b356933a34a0cbac8b6c9065f80a2817303d; archived source capture; exact capture time not independently established
- Inoculation Prompting: Eliciting traits from LLMs during training can suppress them at test-time · 4 AnalysisSelected original resource https://arxiv.org/html/2510.04340; sha256:33651a27c76c3f23dfbdbe6da699b356933a34a0cbac8b6c9065f80a2817303d; archived source capture; exact capture time not independently established
- Inoculation Prompting: Eliciting traits from LLMs during training can suppress them at test-time · 4.5 Inoculated behaviours remain elicitable via promptingSelected original resource https://arxiv.org/html/2510.04340; sha256:33651a27c76c3f23dfbdbe6da699b356933a34a0cbac8b6c9065f80a2817303d; archived source capture; exact capture time not independently established
- Inoculation Prompting: Eliciting traits from LLMs during training can suppress them at test-time · 5 DiscussionSelected original resource https://arxiv.org/html/2510.04340; sha256:33651a27c76c3f23dfbdbe6da699b356933a34a0cbac8b6c9065f80a2817303d; archived source capture; exact capture time not independently established
- Inoculation Prompting: Eliciting traits from LLMs during training can suppress them at test-time · Appendix G Extended LimitationsSelected original resource https://arxiv.org/html/2510.04340; sha256:33651a27c76c3f23dfbdbe6da699b356933a34a0cbac8b6c9065f80a2817303d; archived source capture; exact capture time not independently established
Source checking, extraction, editorial approval and replication are recorded separately. Methodology →
Institutions and roles
Authors (7)
Daniel Tan · Anders Woodruff · Niels Warncke · Arun Jose · Maxime Riché · David Demitri Africa · Mia Taylor
Cite and reuse
Cite the original publication for its findings. Cite this NUWA record separately for its explanation and retained snapshot.
Original publication · BibTeX
@misc{inoculation-prompting,
title = {Inoculation Prompting: Eliciting traits from LLMs during training can suppress them at test-time},
url = {https://www.aisi.gov.uk/research/inoculation-prompting-eliciting-traits-from-llms-during-training-can-suppress-them-at-test-time},
author = {Daniel Tan and Anders Woodruff and Niels Warncke and Arun Jose and Maxime Riché and David Demitri Africa and Mia Taylor},
year = {2025},
date = {2025-10-05},
version = {Source capture 2026-10-25},
note = {Source capture 2026-10-25}
}
NUWA record · BibTeX
@misc{inoculation-prompting-nuwa,
title = {Inoculation Prompting: Eliciting traits from LLMs during training can suppress them at test-time — NUWA research record},
url = {https://whitzard.tech/en/nuwa/whitzard-index/work/inoculation-prompting/},
version = {2026-10-29.1},
organization = {NUWA Frontier AI Safety Lab},
howpublished = {Retained note edition: https://whitzard.tech/nuwa/whitzard-index/data/record-editions/2026-10-29.1--inoculation-prompting.json},
note = {NUWA research note. Source-reading scope and independent review status are stated on the record. Snapshot 2026-10-29.1. Retained note edition: https://whitzard.tech/nuwa/whitzard-index/data/record-editions/2026-10-29.1--inoculation-prompting.json.}
}
Evidence snapshot 2026-10-29.1 · Index updates · RSS
References
- [1] Daniel Tan, Anders Woodruff, et al.. Inoculation Prompting: Eliciting traits from LLMs during training can suppress them at test-time. 2025-10-05 ; 2 Inoculation Prompting; 3 Further Applications; 4 Analysis
- [2] Daniel Tan, Anders Woodruff, et al.. Inoculation Prompting: Eliciting traits from LLMs during training can suppress them at test-time ; Author affiliations on the original paper title page