What AI bias means, and what these tests can tell you.

Bias can mean several different things. Which of them our data can show, which it cannot, and how we try to keep our own selection fair.

This guide separates what our data can show from what it cannot. Facts come from our datasets and are linked to their sources on each model page.

The basics, and what our data shows

“Bias” means several different things

• Political or social leaning: whether answers lean one way on contested topics.
• Stereotyping: whether a model treats groups of people unequally.
• Uneven performance: whether a model is better at some languages, topics or kinds of request than others.
• Flattery (sycophancy): agreeing with you too readily, even when you are wrong.
• Safety behaviour: what a model refuses, and how reliably.
• Bias in the test: for example, people voting for answers that look longer or more polished.

What our data can show

• Uneven performance. Arena breaks its votes down by language, kind of request and topic. Each model page shows where it is relatively stronger or weaker than its overall position.
• For example, MiMo-V2.6-Flash ranks 31st of 77 on English prompts but 58th of 77 on prompts in other languages. That is the biggest English versus other-language gap among the listed models.
• A known skew in the votes. Arena notes that style control, which adjusts for things like answer length and formatting, became its default for text in May 2025. The Arena data we use is the version without style control, so our preference scores are not adjusted for style.

What our data cannot show

None of our three sources measures political lean, stereotyping, flattery or safety behaviour. We cannot say how any model does on those, and nothing on this site should be read as a statement about them.

Bias in this site

• Which models are listed is decided by a published rule, not by hand: 77 models from 16 makers. The largest maker is Anthropic with 12. The rule favours models that Arena's voters and Epoch's runs already cover.
• An earlier draft of this site listed only 15 hand-picked models and was skewed. It was replaced by the rule.
• This site was built with an AI assistant made by Anthropic, whose models appear in the data. The rules and data are published so you can check them.

What this means for you

Strong scores on a general test are not evidence of fairness, and a model that does well in English may do less well in your language. If one of these issues matters to you, test the models on prompts from your own situation, and treat any single score, including ours, as a starting point.

What to do next

Open a model page to see its relative strengths and weaknesses, or find your AI by job.