RESEARCH RECORD

Resolution Diagnostics for Paired LLM Evaluation

Anany Kotawala · Princeton University

Some displayed model gaps remain unresolved under the specified error and power targets; sample size alone is insufficient without paired variance and dependence assumptions.

Research question

Can the available benchmark sample resolve a reported model-performance gap?

Design & assumptions

The analysis inverts paired tests into required sample sizes and examines real per-question leaderboard data with clustering and multiplicity checks.

Findings

Some displayed model gaps remain unresolved under the specified error and power targets; sample size alone is insufficient without paired variance and dependence assumptions.

Limits & counterevidence

The empirical evidence is primarily binary accuracy and close comparisons. It does not show that all rankings are meaningless, nor does a non-significant gap prove equivalence.

Source evidence

No structured numerical extraction is available in this reading package.

Reading scope & review

Selected original HTML sections and table context; not a complete paper/appendix audit or independent reproduction.

Model-assisted reading; independent evidence review pending.

Check the original source ↗