> ## Content Index
> Fetch the complete content index at: https://aiadoption.org/llms.txt
> Use this file to discover other available public pages before exploring further.

# GPT-5 scored 99.8% on standard Slovenian but 88.6% on the Cerkno dialect; 8 of 12 models fell below 60%
- URL: https://aiadoption.org/data/slovenian-dialect-llm-accuracy/
- Published: 2026-10-19T17:08:10.000Z
- Updated: 2026-10-20T22:56:20.000Z
- Description: All 12 models tested scored lower on the Cerkno dialect than on standard Slovenian. GPT-5 fell from 99.8% to 88.6%; Mistral Medium 3.1 from 90.0% to 53.2%.
- Author: Manu
- Tags: Data, AI Benchmarks, Large Language Models, AI in Europe, #chart, #ai-index-2026, #hub-benchmarks, #country-europe

88.6%

GPT-5's accuracy on the Cerkno dialect, the highest of 12 models, against 99.8% on standard Slovenian

GPT-5 accuracy

Cerkno dialect88.6%

Standard Slovenian99.8%

## What the chart shows

DIALECT-COPA tests commonsense reasoning in standard Slovenian and in the Cerkno dialect. All 12 models tested scored lower on the dialect. GPT-5 scored 99.8% in standard Slovenian and 88.6% in the dialect, the best dialect score, followed by Gemini 2.5 Pro (86.4%).

Most other models dropped further: Mistral Medium 3.1 fell from 90.0% to 53.2% and Llama 3.3 from 87.0% to 53.6%. Eight of the 12 scored below 60% on the dialect, close to the 50% expected from guessing between two answers.

## Key numbers

| Measure                                      | Value |
| -------------------------------------------- | ----- |
| GPT-5, standard Slovenian                    | 99.8% |
| GPT-5, Cerkno dialect                        | 88.6% |
| Gemini 2.5 Pro, Cerkno dialect               | 86.4% |
| Mistral Medium 3.1, standard Slovenian       | 90.0% |
| Mistral Medium 3.1, Cerkno dialect           | 53.2% |
| DeepSeek-R1-Distill-Qwen-14B, Cerkno dialect | 49.2% |

## How to read it

- COPA (Choice of Plausible Alternatives) asks a model to pick the more plausible of two answers, so random guessing scores about 50%.
- Results cover these 12 models and one dialect. The AI Index notes that dialects differ from standard languages in spelling, vocabulary and grammar and are rarely represented in training data.
- Model names are shortened; the CSV gives versions (for example gpt-5-2025-08-07).

**Source:** [Slovene DIALECT-COPA benchmark leaderboard](https://aiadoption.org/sources/#slobench-dialect-copa), 2026, via the AI Index Report 2026 (Stanford HAI), figure 3.7.7: *Slobench: accuracy*. Chart and text by AI Adoption.  
Numbers checked against the published data on 20 October 2026.  
[Download the data (CSV)](https://charts.aiadoption.org/ai-index/csv/slovenian-dialect-llm-accuracy.csv?ref=aiadoption.org) · [Cite this chart](#source)