Chat GPT 5, released in August 2025, has produced a range of entertaining hallucinations that went viral, including an image of US presidents, including “Richard Ninun” (instead of Nixon), seemingly born in 1969 but died in 1974, just five years later (Nixon was actually born in 1913 and died in 1994). The very same image had a host of other errors, such as Ronald “Beagan” (Reagan) being born in 1661. A different one, also error-strewn, can be seen here. A similar list of Canadian prime ministers was also flawed.
I wondered whether these had somehow been faked, or maybe they were just temporary blips, so on 14th September 2025 I asked the top AI models all the same question myself.
“Can you produce a picture showing all the US presidents, with their portraits, names and dates when they held office?”
This was what Anthropic’s Claude managed to produce on September 14th 2025 when I gave it this prompt:
ChatGPT was no better, this being produced the same day.
Amongst the obvious errors, the Thomas Jefferson dates are absurd, as are those of Abraham Lincoln, and more. The portraits and dates are not great, either, e.g. the one of “John F Kennedy, 1881-1888”.
Google’s Gemini produced an even worse result. It is a little hard to read without zooming in, but even the title “Unates of America”, does not inspire confidence.
Grok could only manage unreadable gibberish. Bear in mind that these are the leading AIs of the day, and none of their answers would exactly pass muster in a history class.
Bear in mind that there is a great deal of public data about US presidents. Now imagine how inaccurate LLMs might be if asked to provide information on more obscure topics that they do not have much training data for.
We know that hallucinations afflict large language models (LLMs) at a rate that varies, but averages maybe one in five answers. As can be seen from the above, this is common to all the leading LLMs, and the hallucination rate is actually worsening, not getting better as you might hope from a newer software release.
The serious point here is that these are the models that are being implemented by corporations right now, being embedded in serious business applications. Hallucinations are an inherent part of the way that LLMs work: “part of the magic”, according to Sam Altman, CEO of OpenAI. Hallucinations may well be part of the reason why 95% of AI projects are currently failing, according to an MIT study.
Most companies seem to be pressing on regardless, either unaware of the issue or motivated by “fear of missing out” on the benefits that generative AI vendors and the highly paid consultants who implement these projects are promising them. There seems to be no end of AI boosters promoting the technology, as anyone who spends a few minutes on LinkedIn will quickly discover. While there are undoubtedly many genuine uses for generative AI, companies must step back and consider which of their use cases are suitable for this technology and which are not. In some cases, other AI approaches, such as old-fashioned machine learning models, may be more suitable. In other cases, AI may not be the best approach at all. The sooner that we improve AI education so that more people understand the ways LLMs work, and can better select the best use cases and avoid the risky ones, the better the outcome will be.
