> ## Content Index
> Fetch the complete content index at: https://aiadoption.org/llms.txt
> Use this file to discover other available public pages before exploring further.

# No model reached the 74.4% human baseline on Video-MMMU; the best, Keye-VL-1.5-8B, scored 66%
- URL: https://aiadoption.org/data/video-mmmu-leaderboard/
- Published: 2026-10-19T15:55:36.000Z
- Updated: 2026-10-21T06:02:56.000Z
- Description: Keye-VL-1.5-8B scored 66% on Video-MMMU, the best of 25 models, against a human baseline of 74.4%. Claude-3.5-Sonnet was close behind at 65.8%.
- Author: Manu
- Tags: Data, AI Benchmarks, Computer Vision, #chart, #ai-index-2026, #hub-benchmarks

66%

#1 of 25

Video-MMMU accuracy of Keye-VL-1.5-8B (2025), the highest of 25 models; the human baseline is 74.4%

Keye-VL-1.5-8B (2025)66%

Claude-3.5-Sonnet (2024)65.8%

Kimi-VL-A3B-Thinking-2506 (2025)65.2%

## What the chart shows

Video-MMMU tests whether models can learn from 300 expert-level lecture videos across six disciplines, with questions on perception, comprehension and applying the knowledge to new problems. Keye-VL-1.5-8B scored 66% overall, the best of the 25 models listed, ahead of Claude-3.5-Sonnet (65.8%) and Kimi-VL-A3B-Thinking-2506 (65.2%).

None reached the benchmark’s human baseline of 74.4%. The lowest score, VILA1.5-8B’s 20.9%, is 45.1 points below the leader.

## Key numbers

| Measure                          | Value  |
| -------------------------------- | ------ |
| Keye-VL-1.5-8B (2025)            | 66%    |
| Claude-3.5-Sonnet (2024)         | 65.78% |
| Kimi-VL-A3B-Thinking-2506 (2025) | 65.22% |
| GPT-4o (2024)                    | 61.22% |
| VILA1.5-8B (2024)                | 20.89% |
| Human baseline (report)          | 74.44% |

## How to read it

- The 74.4% human baseline comes from the report’s chart; it is not in the data file and is drawn as a dashed line.
- Years follow the report’s grouping of leaderboard entries into 2024 and 2025.
- Overall accuracy covers perception, comprehension and adaptation questions.

**Source:** [Video-MMMU Leaderboard](https://aiadoption.org/sources/#video-mmmu), 2026, via the AI Index Report 2026 (Stanford HAI), figure 2.3.2: *Video-MMMU: overall accuracy*. Chart and text by AI Adoption.  
Numbers checked against the published data on 20 October 2026.  
[Download the data (CSV)](https://charts.aiadoption.org/ai-index/csv/video-mmmu-leaderboard.csv?ref=aiadoption.org) · [Cite this chart](#source)