{"schemaVersion":"nuwa-current-record-1","version":"2026-10-29.1","publication":"published","work":{"id":"abstentionbench","title":"AbstentionBench: Reasoning LLMs Fail on Unanswerable Questions","aliases":["AbstentionBench"],"authors":[],"primaryFunction":"evaluation","secondaryFunctions":[],"kind":"benchmark","sourceId":"abstentionbench-primary","attributionSourceId":"abstentionbench-primary","originalUrl":"https://arxiv.org/abs/2506.09038","publicationDate":"2025-06-10","updatedOn":"2026-10-25","areas":[],"scopeNote":{"en":"Mapped to the documented research question. A risk-topic association is not a severity score.","zh":"依据明确研究问题关联；风险主题不代表严重性评分。"},"protocolIds":["abstentionbench"],"contributions":[],"summary":{"en":"When does a model abstain from questions that cannot be answered reliably?","zh":"模型何时会对无法可靠回答的问题选择弃答？"},"note":{"question":{"en":"When does a model abstain from questions that cannot be answered reliably?","zh":"模型何时会对无法可靠回答的问题选择弃答？"},"method":{"en":"Tests abstention across 20 datasets with unknown, underspecified, false-premise, subjective or outdated questions.","zh":"在 20 个数据集中检验未知答案、信息不足、错误前提、主观或过时问题的弃答。"},"finding":null,"supports":null,"limits":{"en":"Abstention is distinct from a safety refusal and does not alone establish intent or downstream harm.","zh":"弃答不同于安全拒答，不能单独证明意图或下游伤害。"},"connections":null},"noteAvailability":{"en":"This record describes the protocol. Report-specific outcomes are linked through assessments; no unsourced general finding is added.","zh":"本档案说明协议。报告特定结果通过评估记录关联，不补写无出处的总体发现。"},"resources":[],"tags":[]},"reading":{"id":"abstentionbench","title":"AbstentionBench: Reasoning LLMs Fail on Unanswerable Questions","en":{"question":"Do models know when uncertainty or an ill-posed question warrants abstention?","design":"Twenty datasets span unknown answers, underspecification, false premises, subjectivity and outdated information; twenty frontier models are tested with automatic judging.","findings":["The study reports an average 24% degradation in abstention after reasoning fine-tuning.","Targeted system prompts improve scores but do not solve reliable uncertainty handling."],"interpretation":"NUWA interpretation: improved answer accuracy and improved abstention are different training outcomes.","limits":"The stated 24% should not be silently relabeled percentage points; dataset/judge choices constrain generalization.","decisionRelevance":"Model recommendation: retain answerable and unanswerable subsets and evaluate refusal quality separately from correctness.","takeaway":"NUWA interpretation: improved answer accuracy and improved abstention are different training outcomes.","connections":[],"scoring":"Answer/abstain judgments over answerable and unanswerable situations.","conditions":"Twenty datasets and twenty frontier models; system prompts/reasoning interventions vary.","statistics":"The study reports an average 24% degradation in abstention after reasoning fine-tuning.","usage":"Model recommendation: retain answerable and unanswerable subsets and evaluate refusal quality separately from correctness."},"zh":{"question":"不确定或不成立的问题需要弃答时，模型能否识别？","design":"20数据集含未知答案、条件不足、错误前提、主观性和过时信息，以自动判官评价20前沿模型。","findings":["研究报告推理微调后弃答能力平均退化24%。","定向系统提示可提高分数，但未解决可靠不确定性处理。"],"interpretation":"NUWA 解读：正确率提高和弃答改善是不同训练结果。","limits":"所报24%不能擅改为百分点，数据集及判官选择限制外推。","decisionRelevance":"模型建议：保留可答与不可答子集，分别评价弃答质量和正确性。","takeaway":"NUWA 解读：正确率提高和弃答改善是不同训练结果。","connections":[],"scoring":"可答与不可答场景的回答或弃答判断。","conditions":"20数据集20前沿模型，改变系统提示及推理干预。","statistics":"研究报告推理微调后弃答能力平均退化24%。","usage":"模型建议：保留可答与不可答子集，分别评价弃答质量和正确性。"},"assertionPolicy":{"question":"source-research-question","design":"source-finding","findings":"source-finding","interpretation":"nuwa-interpretation","limits":"source-limit-or-explicit-editorial-scope-limit","connections":"source-supported-relationship-only","decisionRelevance":"model-recommendation-not-source-conclusion"},"sourceRefs":[{"sourceId":"abstentionbench-primary","url":"https://arxiv.org/html/2506.09038","locator":"Abstract","version":"Selected original resource https://arxiv.org/html/2506.09038; sha256:3270c69c4a4075490ddef1d8e9a473edb4da508b4dde3b5bcdeeaf4776963942; archived source capture; exact capture time not independently established","artifactSha256":"3270c69c4a4075490ddef1d8e9a473edb4da508b4dde3b5bcdeeaf4776963942"},{"sourceId":"abstentionbench-primary","url":"https://arxiv.org/html/2506.09038","locator":"§3 AbstentionBench","version":"Selected original resource https://arxiv.org/html/2506.09038; sha256:3270c69c4a4075490ddef1d8e9a473edb4da508b4dde3b5bcdeeaf4776963942; archived source capture; exact capture time not independently established","artifactSha256":"3270c69c4a4075490ddef1d8e9a473edb4da508b4dde3b5bcdeeaf4776963942"},{"sourceId":"abstentionbench-primary","url":"https://arxiv.org/html/2506.09038","locator":"§4 Experiments","version":"Selected original resource https://arxiv.org/html/2506.09038; sha256:3270c69c4a4075490ddef1d8e9a473edb4da508b4dde3b5bcdeeaf4776963942; archived source capture; exact capture time not independently established","artifactSha256":"3270c69c4a4075490ddef1d8e9a473edb4da508b4dde3b5bcdeeaf4776963942"},{"sourceId":"abstentionbench-primary","url":"https://arxiv.org/html/2506.09038","locator":"§5 Discussion","version":"Selected original resource https://arxiv.org/html/2506.09038; sha256:3270c69c4a4075490ddef1d8e9a473edb4da508b4dde3b5bcdeeaf4776963942; archived source capture; exact capture time not independently established","artifactSha256":"3270c69c4a4075490ddef1d8e9a473edb4da508b4dde3b5bcdeeaf4776963942"}],"publication":{"state":"published","publishedOn":"2026-10-29","authorizationReference":"user-authorized-global-index-publication-2026-10-29","reviewedCandidateId":"local-6efe31bc4a0c0f37"},"review":{"state":"published-note","sourceScope":"full-text","availableScope":null,"readingCoverage":"selected-source-sections; not a complete section-by-section paper audit","completePaperAudit":false,"independentEvidenceReview":"pending-human-review","bilingualEditorialReview":"pending-human-review","authorship":"model-assisted-research-note","preparedFingerprint":"49069abc664a706bf1f67b10e7aeb2a0d7a55ec535d8fe9ac52e7b032d7356d0"}},"sources":[{"id":"abstentionbench-primary","title":"AbstentionBench","url":"https://arxiv.org/abs/2506.09038","version":"2026-10-25 source capture","publicationDate":"2025-06-10","datePrecision":"day","type":"original-report","verifiedOn":"2026-10-25","authors":[],"documentId":"arxiv:2506.09038"}]}