WhitzardNUWA Lab ↗
NUWANUWA INDEX

Search NUWA Index

Research · Benchmarks · Findings · Perspectives

中文
← NUWA Index

Evidence record

Llama-3.1-70B-Instruct

JudgeBench: A Benchmark for Evaluating LLM-based Judges · Evidence record

University of California, Berkeley / arXiv:2410.12784v2 / Capability

Sources and provenance ↓ Read the research narrative →

Original source charts and materials

Reported results

Source order · reported configurations

Overall accuracy

0–100%
Llama-3.1-70B-Instruct
52.29%
Showing 1 of 1 systems

Values describe the measured outcome under the source protocol, not a general safety score.

Table 2: accuracy of the Arena-Hard judge using different underlying models. This measures judging ability, not a risk rate. Evaluation date not specified; publication / record dates are shown below.

Evaluation conditions and system configurations
Llama-3.1-70B-Instruct
See the source protocol; no independent trial count inferred. · Table 2 · JudgeBench: A Benchmark for Evaluating LLM-based Judges
modelVersion: not specified · harness: not specified · tools: not specified · budget: not specified
Reported results · source order
System / configurationOverall accuracyKnowledgeReasoningMathCodingPublication / verification
Llama-3.1-70B-InstructHarness not specified52.29%51.30%48.98%60.71%52.38%Source checked2026-10-25

Method and scope

Evaluates model-based judges on challenging response comparisons to examine reliability of automated evaluation.

Evidence type
Benchmark
Assessment subject
Model or agent system in the original evaluation configuration
Primary dimension
Capability
Evidence snapshot
2026-10-29.1

Classified by the operational outcome in the original protocol. Topic associations are documented separately from question-level matrix claims.

Source finding

Source-reported measurements for Llama-3.1-70B-Instruct in Table 2; values are transcribed, not independently reproduced.

Limitations

Judge performance is methodological context; it does not directly measure a frontier-risk capability.

Configuration, task and aggregation differences remain as reported. Source order is retained. Transcription is not independent replication; no cross-protocol ranking is supported.

Related research questions

References

Numerical results are reported by the cited authors.

  1. [1] Sijun Tan, Siyuan Zhuang, et al.. JudgeBench: A Benchmark for Evaluating LLM-based Judges. 2024-10-16. v2 [arXiv]; Author affiliations on the original paper title page; Table 2