0.53
AstaBench average score of Asta v0 (Claude Sonnet 4 plus six other models) at $3.40 per problem; ReAct with o3 scored 0.39 for $0.16

What the chart shows

AstaBench tests AI agents on more than 2,400 scientific research problems, from reading the literature to writing code, analysing data and making discoveries. Ai2’s Asta v0 agent, which combines Claude Sonnet 4 with six other models, scored 0.53 on average at $3.40 per problem: the highest score and the highest cost of the 18 entries here.

The next best was a ReAct agent running GPT-5, at 0.44 for $0.31. Score and cost do not move together: ReAct with o3 scored 0.39 for $0.16, while Smolagents Coder with Claude Sonnet 4 cost $1.02 for 0.38. Llama 4 Scout scored lowest, at 0.08 and 0.11.

Key numbers

MeasureValue
Asta v0 (Claude Sonnet 4 + 6 models)0.530 · $3.40
ReAct, GPT-50.440 · $0.31
ReAct, Claude Sonnet 40.401 · $0.40
ReAct, o30.394 · $0.16
Smolagents Coder, Claude Sonnet 40.381 · $1.02
ReAct, Llama 4 Scout 17B0.079 · $0.68

How to read it

  • Scores are averages across AstaBench’s problem sets on a 0–1 scale; cost is the average cost per problem in US dollars.
  • These are the 18 entries in the AI Index’s data, all submitted by Ai2; the AI Index says the full leaderboard covers 57 agents. Each point is one agent framework running one model; hover over a point to see the model.
  • Model dates in labels are the versions listed on the leaderboard.
Source: AstaBench Leaderboard, 2026, via the AI Index Report 2026 (Stanford HAI), figure 5.3.1: AstaBench: average score. Chart and text by AI Adoption.
Numbers checked against the published data on 20 October 2026.
Download the data (CSV) · Cite this chart