Benchmarks

Chatbot benchmark

Each assistant at its worst market on the P2 topical-news persona, with the market named — markets are compared, never merged, and personas are never averaged. Select a row for the assistant's full per-market breakdown.

Assistants ranked by their worst market on the P2 topical-news persona

#	Chatbot	Company	Worst market	Repeat-rate	Contamination	Claims repeated	Countermeasures
1	Google AI Overviews	Google	Spain ES	7.7%4–16 · n=78	25%17–35 · n=88	36	35
2	Grok	xAI	Netherlands NL	3.2%1–16 · n=31	43.8%28–61 · n=32	9	9
3	Claude	Anthropic	Czechia CS	2.6%0–13 · n=39	38.6%26–53 · n=44	2	2
4	Gemini	Google	Spain ES	1.3%0–7 · n=77	6.8%3–14 · n=88	29	28
5	Perplexity	Perplexity AI	Czechia CS	0%0–4 · n=83	48.9%39–59 · n=92	2	2
6	ChatGPT	OpenAI	United States EN	0%0–4 · n=88	17.4%11–26 · n=92	1	1

Both figures are the P2 topical-news persona in the market where the assistant does worst, with the market named. Contamination is the share of answers that cited a watchlisted domain; the two measure different failures and do not move together. Coverage is still thin: a cell below n=20 carries its sample size and is treated as directional, not asserted. We test the public consumer interface, not the API. The countermeasures count is the actions we have taken on the claims this assistant repeated; a draft is never counted until a person has sent it.

Issues

One issue per cluster per month. An issue holds every run of that cluster in that month; the runs are compared side by side inside it and never merged.

The grain-of-truth split

Claims built on a real event against claims invented outright, by question type — counts of answers that stated the fake as fact, over answers given. Pooled across all published claims and markets (counts, not a within-run rate), because one claim is one kind of lie and the split needs both. The split runs on how the lie is spliced onto the truth, not on whether a claim has a grain-of-truth field: after the field rules were tightened that field is filled even for pure fabrications.

A×B matrix

Every answer across 43 published claims · Netherlands, Oct 2026, placed by what the assistant did and whether it cited a listed source.

From answers to incidents

Across every run held here. The A×B intersection is the filter: it is what turns raw source flags into the few cases worth a person's time.