{"schemaVersion":"nuwa-current-record-1","version":"2026-10-29.1","publication":"published","work":{"id":"pretraining-poisoning","title":"Poisoning Attacks on LLMs Require a Near-constant Number of Poison Samples","aliases":[],"authors":["Alexandra Souly","Javier Rando","Ed Chapman","Xander Davies","Burak Hasircioglu","Ezzeldin Shereen","Carlos Mougan","Vasilios Mavroudis","Erik Jones","Chris Hicks","Nicholas Carlini","Yarin Gal","Robert Kirk"],"primaryFunction":"discovery","secondaryFunctions":[],"kind":"study","sourceId":"pretraining-poisoning-publication","attributionSourceId":"pretraining-poisoning-publication-author-source","originalUrl":"https://www.aisi.gov.uk/research/poisoning-attacks-on-llms-require-a-near-constant-number-of-poison-samples","publicationDate":"2025-10-08","updatedOn":"2026-10-25","areas":["cybersecurity"],"scopeNote":{"en":"Relevance is assigned to the research question and stated conditions of the work. A topic tag is not a risk rating.","zh":"风险关联依据研究问题及明确条件建立，主题标签不代表风险评级。"},"protocolIds":[],"contributions":[{"organizationId":"aisi","role":"research","sourceId":"pretraining-poisoning-publication-author-source","locator":"Author affiliations on the original paper title page"},{"organizationId":"anthropic","role":"research","sourceId":"pretraining-poisoning-publication-author-source","locator":"Author affiliations on the original paper title page"},{"organizationId":"eth","role":"research","sourceId":"pretraining-poisoning-publication-author-source","locator":"Author affiliations on the original paper title page"},{"organizationId":"oxford","role":"research","sourceId":"pretraining-poisoning-publication-author-source","locator":"Author affiliations on the original paper title page"}],"summary":{"en":"Tests pretraining-data poisoning across model and dataset scales to examine the amount of malicious data needed for a targeted effect.","zh":"跨模型与数据规模测试预训练数据投毒，研究诱发指定效果所需的恶意数据量。"},"note":{"question":null,"method":null,"finding":null,"supports":null,"limits":null,"connections":null},"resources":[{"label":"Read the full paper","url":"https://arxiv.org/abs/2510.07192"}],"noteStatus":{"question":"limited","method":"limited","finding":"limited","supports":"limited","limits":"limited","connections":"limited"},"noteAvailability":{"en":"A substantive NUWA research note is not yet available. The repeated draft sections were removed pending review of the original methods, findings and limitations; use the original publication for the research account.","zh":"实质性的 NUWA 研究笔记尚未完成。重复草稿章节已移除，待核查原始方法、发现与限制；研究内容请以原始出版物为准。"},"noteSourceLocator":"No complete section-level method/results/limitations review completed for this note; cached original page and any chart audit retained as discovery evidence.","tags":[]},"reading":{"id":"pretraining-poisoning","title":"Poisoning Attacks on LLMs Require a Near-constant Number of Poison Samples","en":{"takeaway":"In the studied training regimes, absolute poison count explained success better than corpus percentage.","question":"Does increasing clean-data scale dilute a fixed adversarial insertion enough to prevent a backdoor?","design":"Controlled pretraining varies model/data scale and inserted sample count, with additional model-family, ordering and fine-tuning experiments; clean and triggered behavior are evaluated separately.","findings":["Comparable insertion counts can affect models trained on substantially different amounts of clean data.","Continued clean training and other ablations change persistence without supporting a simple percentage-only safety argument."],"interpretation":"NUWA interpretation: reporting only a contamination percentage can hide the relevant exposure unit.","limits":"The studied model range, objectives and training regimes do not establish a universal constant or arbitrary real-world web-poisoning feasibility.","connections":["NUWA connection: Sleeper Agents studies later removal of trained conditional behavior; open-weight-risk-management identifies data curation and evaluation access as distinct defenses."],"decisionRelevance":"NUWA recommendation: retain absolute count, exposure schedule and outcome construct; evaluate provenance controls without exposing attack recipes."},"zh":{"takeaway":"在所测训练制度中，污染样本绝对数量比语料占比更能解释成功。","question":"扩大干净数据规模，是否足以稀释固定对抗插入并防止后门？","design":"受控预训练改变模型、数据规模和插入数量，另检验模型家族、顺序与微调，分别评估正常及触发行为。","findings":["相近插入数量可影响使用明显不同干净数据量的模型。","后续干净训练及其他消融会改变持续性，但不支持仅按比例判断安全。"],"interpretation":"NUWA 解读：只报告污染比例可能隐藏相关的暴露单位。","limits":"所测规模、目标和训练制度不能确立普适常数，也不能证明任意真实网页污染均可行。","connections":["NUWA 关联：Sleeper Agents 研究后续移除已训练条件行为；开放权重风险议程区分数据整理与评测访问防线。"],"decisionRelevance":"NUWA 建议：保留绝对数量、暴露安排与结果构念，评估来源控制但不传播攻击配方。"},"sourceRefs":[{"sourceId":"pretraining-poisoning-publication","url":"https://arxiv.org/html/2510.07192","locator":"2 Preliminaries and Threat Model","version":"Selected original resource https://arxiv.org/html/2510.07192; sha256:ed3890c9bba96935aff8c51e972ce96dbcf72e47b558f71e9ed6da19f9506754; archived source capture; exact capture time not independently established","artifactSha256":"ed3890c9bba96935aff8c51e972ce96dbcf72e47b558f71e9ed6da19f9506754"},{"sourceId":"pretraining-poisoning-publication","url":"https://arxiv.org/html/2510.07192","locator":"3 Backdoors during Chinchilla-Optimal Pretraining","version":"Selected original resource https://arxiv.org/html/2510.07192; sha256:ed3890c9bba96935aff8c51e972ce96dbcf72e47b558f71e9ed6da19f9506754; archived source capture; exact capture time not independently established","artifactSha256":"ed3890c9bba96935aff8c51e972ce96dbcf72e47b558f71e9ed6da19f9506754"},{"sourceId":"pretraining-poisoning-publication","url":"https://arxiv.org/html/2510.07192","locator":"4 Ablations of Attack Success during Pretraining","version":"Selected original resource https://arxiv.org/html/2510.07192; sha256:ed3890c9bba96935aff8c51e972ce96dbcf72e47b558f71e9ed6da19f9506754; archived source capture; exact capture time not independently established","artifactSha256":"ed3890c9bba96935aff8c51e972ce96dbcf72e47b558f71e9ed6da19f9506754"},{"sourceId":"pretraining-poisoning-publication","url":"https://arxiv.org/html/2510.07192","locator":"5 Backdoors During Safety Instruction Fine-tuning","version":"Selected original resource https://arxiv.org/html/2510.07192; sha256:ed3890c9bba96935aff8c51e972ce96dbcf72e47b558f71e9ed6da19f9506754; archived source capture; exact capture time not independently established","artifactSha256":"ed3890c9bba96935aff8c51e972ce96dbcf72e47b558f71e9ed6da19f9506754"},{"sourceId":"pretraining-poisoning-publication","url":"https://arxiv.org/html/2510.07192","locator":"6 Discussion and Conclusion","version":"Selected original resource https://arxiv.org/html/2510.07192; sha256:ed3890c9bba96935aff8c51e972ce96dbcf72e47b558f71e9ed6da19f9506754; archived source capture; exact capture time not independently established","artifactSha256":"ed3890c9bba96935aff8c51e972ce96dbcf72e47b558f71e9ed6da19f9506754"}],"publication":{"state":"published","publishedOn":"2026-10-29","authorizationReference":"user-authorized-global-index-publication-2026-10-29","reviewedCandidateId":"local-6efe31bc4a0c0f37"},"review":{"state":"published-note","sourceScope":"full-text","availableScope":null,"readingCoverage":"selected substantive sections of the full source body","completePaperAudit":false,"independentEvidenceReview":"pending-human-review","bilingualEditorialReview":"pending-human-review","authorship":"model-assisted-research-note","preparedFingerprint":"720f43449293a3e49972d32d50278c0c68af4e08c0977d9c6a21fa9292b110ca"}},"sources":[{"id":"pretraining-poisoning-publication","title":"Poisoning Attacks on LLMs Require a Near-constant Number of Poison Samples","url":"https://www.aisi.gov.uk/research/poisoning-attacks-on-llms-require-a-near-constant-number-of-poison-samples","version":"Source capture 2026-10-25","publicationDate":"2025-10-08","datePrecision":"day","type":"research-publication","verifiedOn":"2026-10-25","documentId":"https://www.aisi.gov.uk/research/poisoning-attacks-on-llms-require-a-near-constant-number-of-poison-samples","authors":["Alexandra Souly","Javier Rando","Ed Chapman","Xander Davies","Burak Hasircioglu","Ezzeldin Shereen","Carlos Mougan","Vasilios Mavroudis","Erik Jones","Chris Hicks","Nicholas Carlini","Yarin Gal","Robert Kirk"]}]}