NUWA OPINION
AI Agents That Matter
Original publication ↗ · BibTeX · RISTheir central position is that agent evaluations should control costs, distinguish model research from downstream procurement, and test generalization with meaningful held-out tasks.
Research question
What makes an agent benchmark useful for research and deployment decisions?
Design & assumptions
The authors examine accuracy-cost tradeoffs, reproduce agent comparisons, and analyze holdouts and evaluation standardization.
Argument & evidence
Their central position is that agent evaluations should control costs, distinguish model research from downstream procurement, and test generalization with meaningful held-out tasks.
Limits & counterevidence
These are evidence-informed methodological recommendations, not a frontier-risk score. Cost depends on usage and pricing; the illustrated comparisons do not rank current systems.
Source evidence
No structured numerical extraction is available in this reading package.
Reading scope & review
Selected original HTML sections and table context; not a complete paper/appendix audit or independent reproduction.
Model-assisted reading; independent evidence review pending.
Check the original source ↗