风险评测 · 评测基准

TruthfulQA: Measuring how models mimic human falsehoods

NUWA 研究解读 · 独立复核待完成

首次发表: 2021-09-08来源核验: 2026-10-25

原始来源 ↗引用本记录 ↓

NUWA Research Note

研究问题研究设计与假设来源发现 NUWA 解读局限与相反证据研究与决策意义机构与角色原始资源

模型辅助整理 · 选读原始全文或官方文章中的相关章节；尚未完成逐章节审计。

研究速读

NUWA 解读：这测常见错误信念稳健性，不测故意说谎。

研究问题

模型能否避免复述常见人类误解？

研究设计与假设

38类817题针对模仿性错误，并分别评价真实性与信息量。

TruthfulQA — Figure 2 — Figure 2 · 原文结果图，请结合下方的协议范围与局限解读。原图与来源 ↗

来源发现

原始研究最佳模型真实率58%，人类94%。

该历史比较组中最大模型通常较不真实。

NUWA 解读

NUWA 解读：这测常见错误信念稳健性，不测故意说谎。

局限与相反证据

高分不保证专业领域真实；历史规模结果不能外推所有后续模型。

研究与决策意义

模型建议：保留模型代际，区分真实的不确定表达与有信息的正确回答。

精确来源位置

TruthfulQA: Measuring How Models Mimic Human Falsehoods · AbstractSelected original resource https://arxiv.org/html/2109.07958; sha256:c0181fc915933de5c1d5a93995d17bd531bfbc37639d7dfcf8a5eb6d6fae6f0d; archived source capture; exact capture time not independently established
TruthfulQA: Measuring How Models Mimic Human Falsehoods · §2 The TruthfulQA BenchmarkSelected original resource https://arxiv.org/html/2109.07958; sha256:c0181fc915933de5c1d5a93995d17bd531bfbc37639d7dfcf8a5eb6d6fae6f0d; archived source capture; exact capture time not independently established
TruthfulQA: Measuring How Models Mimic Human Falsehoods · §4 ResultsSelected original resource https://arxiv.org/html/2109.07958; sha256:c0181fc915933de5c1d5a93995d17bd531bfbc37639d7dfcf8a5eb6d6fae6f0d; archived source capture; exact capture time not independently established
TruthfulQA: Measuring How Models Mimic Human Falsehoods · §7 ConclusionSelected original resource https://arxiv.org/html/2109.07958; sha256:c0181fc915933de5c1d5a93995d17bd531bfbc37639d7dfcf8a5eb6d6fae6f0d; archived source capture; exact capture time not independently established

来源核查、数据提取、编辑批准与实验复现分别记录。方法说明 →

机构与角色

研究机构牛津大学 OpenAI

作者 (3)

Stephanie Lin · Jacob Hilton · Owain Evans

评测协议

TruthfulQA →

引用与复用

引用研究发现时请引用原始出版物；引用 NUWA 的整理与解释时，单独引用本站记录及对应快照。

BibTeX ↓RIS ↓记录与来源元数据 · JSON ↓阅读覆盖 · JSON 提交更正

原始出版物 · BibTeX

@misc{truthfulqa,
  title = {TruthfulQA: Measuring how models mimic human falsehoods},
  url = {https://truthful.ai/papers/truthfulqa/},
  author = {Stephanie Lin and Jacob Hilton and Owain Evans},
  year = {2021},
  date = {2021-09-08},
  version = {Source capture 2026-10-25},
  note = {Source capture 2026-10-25}
}

NUWA 记录 · BibTeX

@misc{truthfulqa-nuwa,
  title = {TruthfulQA: Measuring how models mimic human falsehoods — NUWA research record},
  url = {https://whitzard.tech/nuwa/whitzard-index/work/truthfulqa/},
  version = {2026-10-29.1},
  organization = {NUWA Frontier AI Safety Lab},
  howpublished = {Retained note edition: https://whitzard.tech/nuwa/whitzard-index/data/record-editions/2026-10-29.1--truthfulqa.json},
  note = {NUWA research note. Source-reading scope and independent review status are stated on the record. Snapshot 2026-10-29.1. Retained note edition: https://whitzard.tech/nuwa/whitzard-index/data/record-editions/2026-10-29.1--truthfulqa.json.}
}

证据快照 2026-10-29.1 · 索引更新 · RSS

参考文献

[1] Stephanie Lin, Jacob Hilton 等. TruthfulQA: Measuring how models mimic human falsehoods. 2021-09-08
[2] Stephanie Lin, Jacob Hilton 等. TruthfulQA: Measuring How Models Mimic Human Falsehoods ; Author affiliations on the original paper title page; Abstract; §2 The TruthfulQA Benchmark
[3] Codex