{"schemaVersion":"nuwa-current-record-1","version":"2026-10-29.1","publication":"published","work":{"id":"posttrainbench","title":"PostTrainBench: Can LLM Agents Automate LLM Post-Training?","aliases":["PostTrainBench and OpenAI PostTrainBench Lite"],"authors":["Ben Rank","Hardik Bhatnagar","Ameya Prabhu","Shira Eisenberg","Karina Nguyen","Matthias Bethge","Maksym Andriushchenko"],"primaryFunction":"evaluation","secondaryFunctions":[],"kind":"benchmark","sourceId":"posttrainbench-publication","attributionSourceId":"posttrainbench-publication-author-source","originalUrl":"https://arxiv.org/abs/2603.08640","publicationDate":"2026-03-09","updatedOn":"2026-10-25","areas":["autonomy"],"scopeNote":{"en":"The main dimension follows the task outcome in the original protocol. Topic relevance is documented in the parent research record; no unreviewed question-level links are inferred.","zh":"主维度依据原协议的任务结果确定。主题关联见研究档案，不推断尚未审阅的问题级关联。"},"protocolIds":["posttrainbench"],"contributions":[{"organizationId":"ellis","role":"research","sourceId":"posttrainbench-publication-author-source","locator":"Author affiliations on the original paper title page"},{"organizationId":"mpi","role":"research","sourceId":"posttrainbench-publication-author-source","locator":"Author affiliations on the original paper title page"},{"organizationId":"tubingen","role":"research","sourceId":"posttrainbench-publication-author-source","locator":"Author affiliations on the original paper title page"},{"organizationId":"thoughtful","role":"research","sourceId":"posttrainbench-publication-author-source","locator":"Author affiliations on the original paper title page"}],"summary":{"en":"Tests whether agents can improve base models through post-training under a fixed compute budget.","zh":"评测智能体能否在固定计算预算内，通过后训练改进基础模型。"},"note":{"question":null,"method":null,"finding":null,"supports":null,"limits":null,"connections":null},"resources":[],"noteStatus":{"question":"limited","method":"limited","finding":"limited","supports":"limited","limits":"limited","connections":"limited"},"noteAvailability":{"en":"A substantive NUWA research note is not yet available. The repeated draft sections were removed pending review of the original methods, findings and limitations; use the original publication for the research account.","zh":"实质性的 NUWA 研究笔记尚未完成。重复草稿章节已移除，待核查原始方法、发现与限制；研究内容请以原始出版物为准。"},"noteSourceLocator":"No complete section-level method/results/limitations review completed for this note; cached original page and any chart audit retained as discovery evidence.","tags":[]},"reading":{"id":"posttrainbench","title":"PostTrainBench: Can LLM Agents Automate LLM Post-Training?","en":{"question":"Can coding agents autonomously improve a supplied base model through post-training?","design":"Each run pairs one of four small base models with a target benchmark, constrained to ten hours on one H100 and held-out checkpoint evaluation.","findings":["The best reported agent average was 23.2%, against 7.5% base and 51.1% official instruction-tuned averages.","Some targeted model/task pairs exceeded the official counterpart."],"interpretation":"NUWA interpretation: selected post-training gains do not demonstrate general autonomous AI research.","limits":"Single-task optimization, constrained compute and only three frontier-agent runs limit generalization and variance estimates; contamination judgments can err.","decisionRelevance":"Model recommendation: retain target task and checkpoint integrity audits alongside performance gains.","takeaway":"NUWA interpretation: selected post-training gains do not demonstrate general autonomous AI research.","connections":[],"scoring":"Held-out target-benchmark score of the submitted post-trained checkpoint.","conditions":"Ten hours, one H100; target base model and benchmark paired per run.","statistics":"The best reported agent average was 23.2%, against 7.5% base and 51.1% official instruction-tuned averages.","usage":"Model recommendation: retain target task and checkpoint integrity audits alongside performance gains."},"zh":{"question":"编程智能体能否通过后训练自主改善给定基础模型？","design":"四种小基础模型分别配目标基准，每次限单H100十小时，以保留集评价提交检查点。","findings":["最佳智能体平均23.2%，基础7.5%，官方指令模型51.1%。","个别模型任务配对可超过官方版本。"],"interpretation":"NUWA 解读：选定后训练增益不证明通用自主AI研究。","limits":"单任务优化、有限算力和前沿智能体仅三次运行限制外推及方差估计，污染判断可能出错。","decisionRelevance":"模型建议：表现增益旁保留目标任务与检查点完整性审查。","takeaway":"NUWA 解读：选定后训练增益不证明通用自主AI研究。","connections":[],"scoring":"提交后训练检查点的保留目标基准得分。","conditions":"单H100十小时，每次配基础模型与目标基准。","statistics":"最佳智能体平均23.2%，基础7.5%，官方指令模型51.1%。","usage":"模型建议：表现增益旁保留目标任务与检查点完整性审查。"},"assertionPolicy":{"question":"source-research-question","design":"source-finding","findings":"source-finding","interpretation":"nuwa-interpretation","limits":"source-limit-or-explicit-editorial-scope-limit","connections":"source-supported-relationship-only","decisionRelevance":"model-recommendation-not-source-conclusion"},"sourceRefs":[{"sourceId":"posttrainbench-publication-author-source","url":"https://arxiv.org/html/2603.08640","locator":"Abstract","version":"Selected original resource https://arxiv.org/html/2603.08640; sha256:09f5be78dc2b8f33ac7a82fc692485521fd9442d75cca2bd6d3aad76d727d70c; archived source capture; exact capture time not independently established","artifactSha256":"09f5be78dc2b8f33ac7a82fc692485521fd9442d75cca2bd6d3aad76d727d70c"},{"sourceId":"posttrainbench-publication-author-source","url":"https://arxiv.org/html/2603.08640","locator":"§1 Introduction: Our benchmark","version":"Selected original resource https://arxiv.org/html/2603.08640; sha256:09f5be78dc2b8f33ac7a82fc692485521fd9442d75cca2bd6d3aad76d727d70c; archived source capture; exact capture time not independently established","artifactSha256":"09f5be78dc2b8f33ac7a82fc692485521fd9442d75cca2bd6d3aad76d727d70c"},{"sourceId":"posttrainbench-publication-author-source","url":"https://arxiv.org/html/2603.08640","locator":"§3.1 Main Results","version":"Selected original resource https://arxiv.org/html/2603.08640; sha256:09f5be78dc2b8f33ac7a82fc692485521fd9442d75cca2bd6d3aad76d727d70c; archived source capture; exact capture time not independently established","artifactSha256":"09f5be78dc2b8f33ac7a82fc692485521fd9442d75cca2bd6d3aad76d727d70c"},{"sourceId":"posttrainbench-publication-author-source","url":"https://arxiv.org/html/2603.08640","locator":"§7 Discussion: Limitations","version":"Selected original resource https://arxiv.org/html/2603.08640; sha256:09f5be78dc2b8f33ac7a82fc692485521fd9442d75cca2bd6d3aad76d727d70c; archived source capture; exact capture time not independently established","artifactSha256":"09f5be78dc2b8f33ac7a82fc692485521fd9442d75cca2bd6d3aad76d727d70c"}],"publication":{"state":"published","publishedOn":"2026-10-29","authorizationReference":"user-authorized-global-index-publication-2026-10-29","reviewedCandidateId":"local-6efe31bc4a0c0f37"},"review":{"state":"published-note","sourceScope":"full-text","availableScope":null,"readingCoverage":"selected-source-sections; not a complete section-by-section paper audit","completePaperAudit":false,"independentEvidenceReview":"pending-human-review","bilingualEditorialReview":"pending-human-review","authorship":"model-assisted-research-note","preparedFingerprint":"fd7464895ab4b5a48eed9e4860bde5c0c57432f6ec1fd62529a781fef1d6da2f"}},"sources":[{"id":"posttrainbench-publication","title":"PostTrainBench: Can LLM Agents Automate LLM Post-Training?","url":"https://arxiv.org/abs/2603.08640","version":"arXiv:2603.08640v2","publicationDate":"2026-03-09","datePrecision":"day","type":"paper","verifiedOn":"2026-10-25","documentId":"arxiv:2603.08640","authors":["Ben Rank ∗","Hardik Bhatnagar ∗","Ameya Prabhu","Shira Eisenberg †","Nguyen Karina","Matthias Bethge","Maksym Andriushchenko"],"aliases":["https://doi.org/10.48550/arXiv.2603.08640","https://arxiv.org/abs/2603.08640","https://arxiv.org/pdf/2603.08640"]}]}