1.8%
hallucination rate of Ant Group's Finix S1 32B on Vectara's summary test, the lowest of the 15 models shown; Qwen3-14B: 5.4%

What the chart shows

Vectara’s Hughes Hallucination Evaluation Model (HHEM) checks how often a model’s summaries of CNN and Daily Mail news articles include information the article does not support. Of the 15 models the AI Index shows, Ant Group’s Finix S1 32B had the lowest rate, 1.8%, followed by Gemini 2.5 Flash-Lite (3.3%) and Microsoft’s Phi-4 (3.7%).

The other 12 models fell between 4.1% and 5.4%, with Qwen3-14B the highest. The AI Index notes that last year’s leaderboard showed top models at 1.3% to 2.9%, but with a different set of models, so the two years are not directly comparable.

Key numbers

MeasureValue
antgroup/finix_s1_32b1.8%
google/gemini-2.5-flash-lite3.3%
microsoft/Phi-43.7%
meta-llama/Llama-3.3-70B-Instruct-Turbo4.1%
qwen/qwen3-14b5.4%

How to read it

  • HHEM-2.3 is Vectara’s automated judge: it decides whether each summary is consistent with its source article. Its scale is not comparable with the AA-Omniscience hallucination rate (related chart).
  • The chart shows the 15 models the AI Index selected; Vectara’s full leaderboard has more.
  • Model names are shown without the developer prefix the leaderboard lists before a slash (e.g. “antgroup/”); the CSV keeps the full names.
Source: HHEM Leaderboard, 2026, via the AI Index Report 2026 (Stanford HAI), figure 3.2.5: HHEM-2.3: hallucination rate. Chart and text by AI Adoption.
Numbers checked against the published data on 20 October 2026.
Download the data (CSV) · Cite this chart