66%
#1 of 25
Video-MMMU accuracy of Keye-VL-1.5-8B (2025), the highest of 25 models; the human baseline is 74.4%
What the chart shows
Video-MMMU tests whether models can learn from 300 expert-level lecture videos across six disciplines, with questions on perception, comprehension and applying the knowledge to new problems. Keye-VL-1.5-8B scored 66% overall, the best of the 25 models listed, ahead of Claude-3.5-Sonnet (65.8%) and Kimi-VL-A3B-Thinking-2506 (65.2%).
None reached the benchmark’s human baseline of 74.4%. The lowest score, VILA1.5-8B’s 20.9%, is 45.1 points below the leader.
Key numbers
| Measure | Value |
|---|---|
| Keye-VL-1.5-8B (2025) | 66% |
| Claude-3.5-Sonnet (2024) | 65.78% |
| Kimi-VL-A3B-Thinking-2506 (2025) | 65.22% |
| GPT-4o (2024) | 61.22% |
| VILA1.5-8B (2024) | 20.89% |
| Human baseline (report) | 74.44% |
How to read it
- The 74.4% human baseline comes from the report’s chart; it is not in the data file and is drawn as a dashed line.
- Years follow the report’s grouping of leaderboard entries into 2024 and 2025.
- Overall accuracy covers perception, comprehension and adaptation questions.
Source: Video-MMMU Leaderboard, 2026, via the AI Index Report 2026 (Stanford HAI), figure 2.3.2: Video-MMMU: overall accuracy. Chart and text by AI Adoption.
Numbers checked against the published data on 20 October 2026.
Download the data (CSV) · Cite this chart
Numbers checked against the published data on 20 October 2026.
Download the data (CSV) · Cite this chart
Discussion