{"schemaVersion":"nuwa-current-record-1","version":"2026-10-29.1","publication":"published","work":{"id":"rogue-replication-threat","title":"The Rogue Replication Threat Model","aliases":[],"authors":[],"primaryFunction":"governance","secondaryFunctions":[],"kind":"technical-report","sourceId":"rogue-replication-threat-publication","attributionSourceId":"rogue-replication-threat-publication","originalUrl":"https://metr.org/blog/2024-11-12-rogue-replication-threat-model/","publicationDate":"2024-11-12","updatedOn":"2026-10-25","areas":["autonomy"],"scopeNote":{"en":"Relevance is assigned to the research question and stated conditions of the work. A topic tag is not a risk rating.","zh":"风险关联依据研究问题及明确条件建立，主题标签不代表风险评级。"},"protocolIds":[],"contributions":[{"organizationId":"metr","role":"publisher","sourceId":"rogue-replication-threat-publication","locator":"Official publication masthead and article byline"}],"summary":{"en":"Develops a threat model for unauthorized replication and identifies the capabilities and conditions that would enable it.","zh":"为未经授权的复制建立威胁模型，并识别其所需能力与条件。"},"note":{"question":null,"method":null,"finding":null,"supports":null,"limits":null,"connections":null},"resources":[],"noteStatus":{"question":"limited","method":"limited","finding":"limited","supports":"limited","limits":"limited","connections":"limited"},"noteAvailability":{"en":"A substantive NUWA research note is not yet available. The repeated draft sections were removed pending review of the original methods, findings and limitations; use the original publication for the research account.","zh":"实质性的 NUWA 研究笔记尚未完成。重复草稿章节已移除，待核查原始方法、发现与限制；研究内容请以原始出版物为准。"},"noteSourceLocator":"No complete section-level method/results/limitations review completed for this note; cached original page and any chart audit retained as discovery evidence.","tags":[]},"reading":{"id":"rogue-replication-threat","title":"The Rogue Replication Threat Model","en":{"takeaway":"The threat model treats replication as a multi-condition scenario and distinguishes ruling out risk from proving it.","question":"Which prerequisites would make autonomous replication socially consequential, and what would a failed or passed evaluation imply?","design":"METR lays out a conceptual threat sequence and candidate threshold evaluations, using qualitative argument and specialist consultation rather than reporting a completed benchmark campaign.","findings":["The authors are more confident in failure below a threshold as rule-out evidence than success as proof of a large resilient rogue population.","They explicitly deprioritize building the full threshold and redirect attention toward general autonomy and independently concerning capabilities."],"interpretation":"NUWA interpretation: copying a model is only one link in a larger threat argument; capability success does not establish all downstream conditions.","limits":"Resource, competition, shutdown and harm assumptions are uncertain; researchers disagree about key outcomes, and the candidate evaluation list is not a result set.","connections":["NUWA connection: the self-replication studies measure narrower capabilities; RepliBench operationalizes some prerequisites without validating this entire threat story."],"decisionRelevance":"NUWA recommendation: maintain an assumption graph and separate empirical prerequisites, conceptual extrapolation and governance judgment."},"zh":{"takeaway":"威胁模型将复制视为多条件情景，并区分排除风险与证明风险。","question":"哪些前提让自主复制具有社会后果，评测失败或通过分别意味着什么？","design":"METR 提出概念威胁链和候选阈值评测，依据定性论证与专家咨询，而非已完成的基准测试活动。","findings":["作者更有信心把低于阈值的失败用作排除证据，而非将通过视为大规模持久失控群体的证明。","作者明确降低完整阈值建设优先级，转向通用自主性及本身就值得关注的能力。"],"interpretation":"NUWA 解读：复制模型只是更大威胁论证的一环；能力成功不确立所有下游条件。","limits":"资源、竞争、关停和危害假设不确定，研究者对关键结果有分歧，候选评测列表不是结果集。","connections":["NUWA 关联：自复制研究测更窄能力，RepliBench 操作化部分前提但未验证整条威胁故事。"],"decisionRelevance":"NUWA 建议：维护假设图，分别记录实证前提、概念外推与治理判断。"},"sourceRefs":[{"sourceId":"rogue-replication-threat-publication","url":"https://metr.org/blog/2024-11-12-rogue-replication-threat-model/","locator":"Our Conclusions","version":"Selected original resource https://metr.org/blog/2024-11-12-rogue-replication-threat-model/; sha256:10de61143a14636fbbd3ee12d2a953b443179de74ce4b49f83567063c2fac41f; archived source capture; exact capture time not independently established","artifactSha256":"10de61143a14636fbbd3ee12d2a953b443179de74ce4b49f83567063c2fac41f"},{"sourceId":"rogue-replication-threat-publication","url":"https://metr.org/blog/2024-11-12-rogue-replication-threat-model/","locator":"How We Are Orienting to This Threat Story","version":"Selected original resource https://metr.org/blog/2024-11-12-rogue-replication-threat-model/; sha256:10de61143a14636fbbd3ee12d2a953b443179de74ce4b49f83567063c2fac41f; archived source capture; exact capture time not independently established","artifactSha256":"10de61143a14636fbbd3ee12d2a953b443179de74ce4b49f83567063c2fac41f"},{"sourceId":"rogue-replication-threat-publication","url":"https://metr.org/blog/2024-11-12-rogue-replication-threat-model/","locator":"Threshold Evaluations for Rogue Replication: intended evidential role","version":"Selected original resource https://metr.org/blog/2024-11-12-rogue-replication-threat-model/; sha256:10de61143a14636fbbd3ee12d2a953b443179de74ce4b49f83567063c2fac41f; archived source capture; exact capture time not independently established","artifactSha256":"10de61143a14636fbbd3ee12d2a953b443179de74ce4b49f83567063c2fac41f"},{"sourceId":"rogue-replication-threat-publication","url":"https://metr.org/blog/2024-11-12-rogue-replication-threat-model/","locator":"How likely are AI agents that pass this threshold to develop large and robust rogue AI populations?","version":"Selected original resource https://metr.org/blog/2024-11-12-rogue-replication-threat-model/; sha256:10de61143a14636fbbd3ee12d2a953b443179de74ce4b49f83567063c2fac41f; archived source capture; exact capture time not independently established","artifactSha256":"10de61143a14636fbbd3ee12d2a953b443179de74ce4b49f83567063c2fac41f"},{"sourceId":"rogue-replication-threat-publication","url":"https://metr.org/blog/2024-11-12-rogue-replication-threat-model/","locator":"Conclusion","version":"Selected original resource https://metr.org/blog/2024-11-12-rogue-replication-threat-model/; sha256:10de61143a14636fbbd3ee12d2a953b443179de74ce4b49f83567063c2fac41f; archived source capture; exact capture time not independently established","artifactSha256":"10de61143a14636fbbd3ee12d2a953b443179de74ce4b49f83567063c2fac41f"}],"publication":{"state":"published","publishedOn":"2026-10-29","authorizationReference":"user-authorized-global-index-publication-2026-10-29","reviewedCandidateId":"local-6efe31bc4a0c0f37"},"review":{"state":"published-note","sourceScope":"full-text","availableScope":null,"readingCoverage":"selected substantive sections of the full source body","completePaperAudit":false,"independentEvidenceReview":"pending-human-review","bilingualEditorialReview":"pending-human-review","authorship":"model-assisted-research-note","preparedFingerprint":"c8c00d7ec916591d43e17fafcff265e9579b8f798b3644e3b99342a1922d6a7c"}},"sources":[{"id":"rogue-replication-threat-publication","title":"The Rogue Replication Threat Model","url":"https://metr.org/blog/2024-11-12-rogue-replication-threat-model/","version":"Source capture 2026-10-25","publicationDate":"2024-11-12","datePrecision":"day","type":"research-publication","verifiedOn":"2026-10-25","documentId":"https://metr.org/blog/2024-11-12-rogue-replication-threat-model"}]}