15.6%
Δknowledge of GPT-4o on Video-MMMU, the highest of 23 models, against 33.1% for human experts
What the chart shows
Video-MMMU’s Δknowledge measures how much a model’s accuracy on new-problem questions improves after it watches the lecture video, as a share of the room it had to improve. Human experts score 33.1%. GPT-4o is the best of 23 models at 15.6%, under half the human figure, followed by Claude-3.5-Sonnet (11.4%) and Qwen-2.5-VL-72B (9.7%).
Eight models score below zero, meaning they did worse after watching, down to InternVL2-8B at −8.5%. Keye-VL-1.5-8B, the most accurate model overall, shows no gain at all (0.0%).
Key numbers
| Measure | Value |
|---|---|
| Human expert | 33.1% |
| GPT-4o | 15.6% |
| Claude-3.5-Sonnet | 11.4% |
| Keye-VL-1.5-8B | 0.0% |
| InternVL2-8B | −8.5% |
How to read it
- The benchmark defines Δknowledge as (accuracy after the video − accuracy before) ÷ (100% − accuracy before), on its adaptation questions. The report describes the values as percentage points gained; we follow the benchmark’s definition.
- Negative values mean accuracy fell after the model saw the video.
- Covers the 23 models listed on the Video-MMMU leaderboard plus its human expert reference.
Source: Video-MMMU Leaderboard, 2026, via the AI Index Report 2026 (Stanford HAI), figure 2.3.3: Video-MMMU: ∆knowledge. Chart and text by AI Adoption.
Numbers checked against the published data on 20 October 2026.
Download the data (CSV) · Cite this chart
Numbers checked against the published data on 20 October 2026.
Download the data (CSV) · Cite this chart
Discussion