When AI Starts ‘Hallucinating’, You’re Likely Already Along for the Ride

When AI chatbots receive a query, oftentimes, it won’t admit it doesn’t know.

Human shaped robot Ameca of British manufacturer Engineered Arts interacts with visitors in Geneva, Switzerland, on July 6, 2023. Johannes Simon/Getty Images

Imagine for a moment you’re looking for some health tips.

Instead of Googling, you turn to the newest AI chatbot, which makes a simple suggestion: stop eating “sodium chloride” (table salt) and instead eat “sodium bromide”—a substance that can cause toxicity and affect the respiratory, nervous, and hormonal systems.

While it may sound farfetched, this incident actually happened in the United States, where a 60-year-old man became ill after consuming sodium bromide over three months.

He told doctors he had asked ChatGPT about it, which recommended the substance.

To verify the findings, doctors also asked ChatGPT about what should replace table salt in a diet, and the AI program again suggested bromide.

Alarmingly, it did not include health warnings with researchers also finding no sources to trace the conclusion from.

In a separate and seemingly disparate case in October 2025, “Big Four” consultancy firm Deloitte was required to partially refund the cost of a report to the Australian federal government (A$97,000) after it was found generative AI was used, producing recommendations based on references that did not exist.

What are AI Hallucinations?

These occurrences are now termed “AI hallucinations.”

It describes a situation where an AI program fills in knowledge gaps by guessing its own conclusions or misinterpreting data.

Why does this occur?

Well, artificial intelligence despite its name, does not actually have its own “intelligence.”

Instead, it learns language patterns (for example, sentence structures, context, tone, and more) to “train” itself, and does not actually understand the meaning of words.

These conditions create the likelihood of AI programs “hallucinating” and coming up with its own conclusions to the millions of queries people are feeding it daily.

For example, the image below was produced in Microsoft Copilot with the prompt: “Can you draw a map of the world and show all countries that start with the letter ‘S.’”

Microsoft Copilot was asked to create a world map highlighting all the countries beginning with the letter "S."
Microsoft Copilot

Does it Matter?

Paul Darwen, academic lead and AI researcher at James Cook University, said there were two big concerns with AI hallucinations.

“The first and most obvious reason is that people ask the chatbot about a topic they don’t know much about (otherwise they wouldn’t need to ask) then they can’t judge how accurate the answer is, so even if the answer is ludicrously wrong they'll tend to blindly believe it,” he told The Epoch Times.

Darwen says the second less obvious concern is its contributing to dropping learning standards among younger individuals.

He said school students were one of the biggest users of generative AI, pointing to revealing American statistics showing ChatGPT usage drops by around 75 percent during the holiday period.

“This suggests that young people aren’t learning how to read and write, or how to construct a mental model of the world, because they can simply outsource their education to ChatGPT,” Darwen said.

“If we are charitable and maybe some of these students are using the chatbots as an interactive tutor, then the chatbot will sometimes hallucinate and fill their young minds with nonsense.

“This is creating a whole generation of young people a shaky grasp on what’s real in the wider world. But it’s not immediately apparent, because they’re still getting passing grades, but later in life it will lead to some bad decisions.”

How to Stop Society Falling into the Hallucination Trap

As generative AI becomes more advanced and “smarter”, it tends to hallucinate more. Couple this with the prevalence of AI and the issue starts to impact broader society.

This is why, Darwen believes publicising examples of hallucination can help inoculate society against it.

“People tend to have bias towards thinking that they are receiving honest answers, so if they see answers whose validity they can judge, it rams home just how common AI hallucinations are,” Darwen said. 

“Another approach is to ask someone to ask the chatbot a question which only they (the human) knows the answer to, and see if the chatbot says, ‘I don’t know,’ or if it makes up a hallucination instead.”

The academic says chatbots have a tendency to provide answers over admitting it does not know the answer.

Ideally, the solution is to address the problem at its source—training AI models on diverse and well-defined data, placing constraints on the responses AI can give, and more.

However, this may be difficult with current technology.