93
PlanBench tasks DeepSeek R1 solved across eight domains with obfuscated descriptions, down from 157 with standard ones
What the chart shows
In this version of PlanBench the task descriptions are scrambled to disguise their structure. Across eight domains DeepSeek R1 solved 93 tasks, down from 157 with standard descriptions, and GPT-5 solved 152, down from 205. Gemini fell less, from 155 to 146. LAMA’s result is unchanged at 204.
The effect varies by domain. DeepSeek R1 solved no Floortile or Sokoban tasks, and GPT-5 fell from 21 to 12 on Blocksworld. Gemini scored higher in four domains, including Blocksworld (21 to 28) and Spanner (30 to 39).
Key numbers
| Measure | Value |
|---|---|
| DeepSeek R1, all eight domains (standard: 157) | 93 |
| GPT-5, all eight domains (standard: 205) | 152 |
| Gemini, all eight domains (standard: 155) | 146 |
| LAMA, all eight domains | 204 |
| DeepSeek R1, Floortile and Sokoban | 0 |
How to read it
- Totals are sums across the eight domains, calculated by us; standard-description totals come from the related chart.
- LAMA’s scores are identical in both versions of the data.
- Up to 45 tasks per domain. Source study: Corrêa et al. (2025), arXiv 2511.09378.
Source: Corrêa et al., 2025, via the AI Index Report 2026 (Stanford HAI), figure 2.4.10: PlanBench: task solved on obfuscated planning. Chart and text by AI Adoption.
Numbers checked against the published data on 20 October 2026.
Download the data (CSV) · Cite this chart
Numbers checked against the published data on 20 October 2026.
Download the data (CSV) · Cite this chart
Discussion