{"schemaVersion":"nuwa-record-evidence-1","batchId":"research-2026-10-28","baseSnapshotId":"2026-10-27.3","publication":"published","dossier":{"id":"optstop","en":{"takeaway":"Adaptive stopping must preserve rare-event sensitivity and state its uncertainty convention.","question":"When can adaptive stopping save evaluation budget without obscuring rare outcomes?","design":"The method stops sampling according to posterior precision and reports savings across nine validation cells. The stopping rule uses a 97% credible interval; other figures and cells use separately stated uncertainty conventions.","findings":["Planned trial reductions ranged from 57.2% to 97.3% across the nine cells. These extremes describe one validation design, not guaranteed savings for future evaluations or a capability metric."],"interpretation":"Meeting an interval-width target does not guarantee coverage or preserve sensitivity to rare dangerous behaviour. Savings need interpretation with the stopping rule and low-probability performance.","limits":"The main validation configuration is limited. Different interval conventions cannot be silently relabeled as 95% confidence intervals.","connections":[],"decisionRelevance":"NUWA recommendation: Use the existing optstop record to compare planned and realised sampling, with rare-event diagnostics and explicit interval type."},"zh":{"takeaway":"九种验证设置的节省为约 57%–97%；停止规则用 97% 可信区间，另有不同统计区间，不能统一为 95% CI。","question":"何时停止采样，既节省预算又保持精度？","design":"方法按后验精度决定何时停止采样，九个验证单元报告相对计划试验量的节省。停止规则使用 97% 可信区间；文中其他图表和单元的区间约定分别保留。","findings":["九个验证单元相对计划试验量的节省为 57.2%–97.3%。范围描述一套验证设计，不保证未来评测同样节省，也不是模型能力指标。"],"interpretation":"达到区间宽度目标不等于保证覆盖率，也不自动保持对稀有危险行为的检出能力。资源节省必须连同停止规则和低概率区域表现一起解释。","limits":"单一主要验证配置；区间宽度保证不等于覆盖率保证，极端/稀有结果尤需谨慎。","connections":[],"decisionRelevance":"NUWA 建议：补充既有 optstop 条目的停止条件、反例与节省适用范围。"},"assertionPolicy":{"findings":"source-finding","interpretation":"nuwa-interpretation","decisionRelevance":"recommendation"},"observations":[{"id":"optstop-min","sourceId":"dr-20261028-optimal-stopping","metric":"Planned trial reduction","value":57.2,"unit":"%","assertionKind":"source-finding","sourceRefs":[{"sourceId":"dr-20261028-optimal-stopping","url":"https://arxiv.org/html/2608.14425","artifactSha256":"43a40c11b2812c7aa61e8ae9b4cc9f1a5ba6f870af8e92af928cd1960565f137","textSha256":"d45226d941cdc95750fdb9b831c9ae6b1078310f00e41b771497fa61c382d11f","locator":"§3.1; text lines 00188,00707","resourceEdition":"arXiv:2608.14425v1 [cs.AI] 14 Aug 2026","capturedAt":"2026-10-28T07:25:52.921145+00:00","title":"Knowing When to Stop: Bayesian Optimal Stopping for LLM Evaluations","version":"Original capture 2026-10-28T07:25:52.921145+00:00; sha256:43a40c11b2812c7aa61e8ae9b4cc9f1a5ba6f870af8e92af928cd1960565f137"}],"sourceReported":true,"publication":"published","humanEvidenceApproval":null,"comparisonPermission":"descriptive-within-source-only","numerator":null,"denominator":null,"interval":null,"modelId":null,"configuration":{"precisionThreshold":0.05,"stoppingCredibleLevel":0.97,"validation":"nine cells;200items except mid-binary198;10epochs"},"limitations":"Extremal cells across one validation design; not guaranteed future savings or a model capability score."},{"id":"optstop-max","sourceId":"dr-20261028-optimal-stopping","metric":"Planned trial reduction","value":97.3,"unit":"%","assertionKind":"source-finding","sourceRefs":[{"sourceId":"dr-20261028-optimal-stopping","url":"https://arxiv.org/html/2608.14425","artifactSha256":"43a40c11b2812c7aa61e8ae9b4cc9f1a5ba6f870af8e92af928cd1960565f137","textSha256":"d45226d941cdc95750fdb9b831c9ae6b1078310f00e41b771497fa61c382d11f","locator":"§3.1; text lines 00188,00959","resourceEdition":"arXiv:2608.14425v1 [cs.AI] 14 Aug 2026","capturedAt":"2026-10-28T07:25:52.921145+00:00","title":"Knowing When to Stop: Bayesian Optimal Stopping for LLM Evaluations","version":"Original capture 2026-10-28T07:25:52.921145+00:00; sha256:43a40c11b2812c7aa61e8ae9b4cc9f1a5ba6f870af8e92af928cd1960565f137"}],"sourceReported":true,"publication":"published","humanEvidenceApproval":null,"comparisonPermission":"descriptive-within-source-only","numerator":null,"denominator":null,"interval":null,"modelId":null,"configuration":{"precisionThreshold":0.05,"stoppingCredibleLevel":0.97,"validation":"nine cells;200items except mid-binary198;10epochs"},"limitations":"Extremal cells across one validation design; not guaranteed future savings or a model capability score."}],"relatedWorkIds":[],"supplementSourceId":"optimal-stopping","batchId":"research-2026-10-28","sourceRefs":[{"sourceId":"dr-20261028-optimal-stopping","url":"https://arxiv.org/html/2608.14425","title":"Knowing When to Stop: Bayesian Optimal Stopping for LLM Evaluations","locator":"§§2–4; Fig.2–3; Appendix B; text lines 00186,00772","version":"Original resource captured 2026-10-28T07:25:52.921145+00:00; sha256:43a40c11b2812c7aa61e8ae9b4cc9f1a5ba6f870af8e92af928cd1960565f137","artifactSha256":"43a40c11b2812c7aa61e8ae9b4cc9f1a5ba6f870af8e92af928cd1960565f137","textSha256":"d45226d941cdc95750fdb9b831c9ae6b1078310f00e41b771497fa61c382d11f","capturedAt":"2026-10-28T07:25:52.921145+00:00","resourceEdition":"arXiv:2608.14425v1 [cs.AI] 14 Aug 2026"}],"publication":{"state":"published","publishedOn":"2026-10-29","authorizationReference":"user-authorized-global-index-publication-2026-10-29","reviewedCandidateId":"local-6efe31bc4a0c0f37"},"review":{"state":"published-note","sourceScope":"full-text","availableScope":null,"readingCoverage":"selected-original-sections","completePaperAudit":false,"independentEvidenceReview":"pending-human-review","bilingualEditorialReview":"pending-human-review","authorship":"model-assisted-research-note","preparedFingerprint":"337f081f325c2f0ad1a643a9077f115addb550e855b259eee22e21066194ae85"}},"update":{"id":"optstop","review":{"state":"published-note","sourceScope":"full-text","availableScope":null,"readingCoverage":"selected-original-sections","completePaperAudit":false,"independentEvidenceReview":"pending-human-review","bilingualEditorialReview":"pending-human-review","authorship":"model-assisted-research-note","preparedFingerprint":"337f081f325c2f0ad1a643a9077f115addb550e855b259eee22e21066194ae85"},"text":{"en":"The stopping rule uses a 97% credible interval, not a generic 95% confidence interval. Savings of 57.2%–97.3% describe nine validation cells; reaching a width target does not guarantee coverage or rare-behaviour sensitivity.","zh":"停止规则使用 97% 可信区间，不能统一改称 95% 置信区间。九单元的 57.2%–97.3% 节省属于该验证设计；达到宽度目标并不保证覆盖率或稀有行为的检出能力。"},"sourceRefs":[{"sourceId":"dr-20261028-optimal-stopping","url":"https://arxiv.org/html/2608.14425","title":"Knowing When to Stop: Bayesian Optimal Stopping for LLM Evaluations","locator":"§§2–4; Fig.2–3; Appendix B; text lines 00186,00772","version":"Original resource captured 2026-10-28T07:25:52.921145+00:00; sha256:43a40c11b2812c7aa61e8ae9b4cc9f1a5ba6f870af8e92af928cd1960565f137","artifactSha256":"43a40c11b2812c7aa61e8ae9b4cc9f1a5ba6f870af8e92af928cd1960565f137","textSha256":"d45226d941cdc95750fdb9b831c9ae6b1078310f00e41b771497fa61c382d11f","capturedAt":"2026-10-28T07:25:52.921145+00:00","resourceEdition":"arXiv:2608.14425v1 [cs.AI] 14 Aug 2026"}],"relatedWorkIds":[],"publication":{"state":"published","publishedOn":"2026-10-29","authorizationReference":"user-authorized-global-index-publication-2026-10-29","reviewedCandidateId":"local-6efe31bc4a0c0f37"}}}