NUWA RESEARCH RECORD
AI Agents That Matter
Original publication ↗ · BibTeX · RISTheir central position is that agent evaluations should control costs, distinguish model research from downstream procurement, and test generalization with meaningful held-out tasks.
Research question
What makes an agent benchmark useful for research and deployment decisions?
Design & assumptions
The authors examine accuracy-cost tradeoffs, reproduce agent comparisons, and analyze holdouts and evaluation standardization.
Findings
Their central position is that agent evaluations should control costs, distinguish model research from downstream procurement, and test generalization with meaningful held-out tasks.
Limits & counterevidence
These are evidence-informed methodological recommendations, not a frontier-risk score. Cost depends on usage and pricing; the illustrated comparisons do not rank current systems.
Source evidence
No structured numerical extraction is available in this reading package.
Reading scope & review
Selected original HTML sections and table context; not a complete paper/appendix audit or independent reproduction.
Model-assisted reading; independent evidence review pending.
Check the original source ↗