> ## Content Index
> Fetch the complete content index at: https://aiadoption.org/llms.txt
> Use this file to discover other available public pages before exploring further.

# Finix S1 32B hallucinated in 1.8% of document summaries, the lowest of 15 models on Vectara's HHEM test
- URL: https://aiadoption.org/data/llm-hallucination-rates-document-summaries/
- Published: 2026-10-19T17:07:55.000Z
- Updated: 2026-10-21T06:04:37.000Z
- Description: On Vectara's HHEM test, which checks summaries of news articles against the source, Finix S1 32B hallucinated in 1.8% of summaries. Most of the 15 models shown were between 4.1% and 5.4%.
- Author: Manu
- Tags: Data, AI Ethics, Large Language Models, #chart, #ai-index-2026, #hub-ethics

1.8%

hallucination rate of Ant Group's Finix S1 32B on Vectara's summary test, the lowest of the 15 models shown; Qwen3-14B: 5.4%

Hallucination rate (lower is better)

Finix S1 32B1.8%

Gemini 2.5 Flash-Lite3.3%

Phi-43.7%

Qwen3-14B5.4%

## What the chart shows

Vectara’s Hughes Hallucination Evaluation Model (HHEM) checks how often a model’s summaries of CNN and Daily Mail news articles include information the article does not support. Of the 15 models the AI Index shows, Ant Group’s Finix S1 32B had the lowest rate, 1.8%, followed by Gemini 2.5 Flash-Lite (3.3%) and Microsoft’s Phi-4 (3.7%).

The other 12 models fell between 4.1% and 5.4%, with Qwen3-14B the highest. The AI Index notes that last year’s leaderboard showed top models at 1.3% to 2.9%, but with a different set of models, so the two years are not directly comparable.

## Key numbers

| Measure                                 | Value |
| --------------------------------------- | ----- |
| antgroup/finix\_s1\_32b                 | 1.8%  |
| google/gemini-2.5-flash-lite            | 3.3%  |
| microsoft/Phi-4                         | 3.7%  |
| meta-llama/Llama-3.3-70B-Instruct-Turbo | 4.1%  |
| qwen/qwen3-14b                          | 5.4%  |

## How to read it

- HHEM-2.3 is Vectara’s automated judge: it decides whether each summary is consistent with its source article. Its scale is not comparable with the AA-Omniscience hallucination rate ([related chart](https://aiadoption.org/data/llm-hallucination-rates-aa-omniscience/)).
- The chart shows the 15 models the AI Index selected; Vectara’s full leaderboard has more.
- Model names are shown without the developer prefix the leaderboard lists before a slash (e.g. “antgroup/”); the CSV keeps the full names.

**Source:** [HHEM Leaderboard](https://aiadoption.org/sources/#hhem-leaderboard), 2026, via the AI Index Report 2026 (Stanford HAI), figure 3.2.5: *HHEM-2.3: hallucination rate*. Chart and text by AI Adoption.  
Numbers checked against the published data on 20 October 2026.  
[Download the data (CSV)](https://charts.aiadoption.org/ai-index/csv/llm-hallucination-rates-document-summaries.csv?ref=aiadoption.org) · [Cite this chart](#source)