What are AI hallucinations?

An AI hallucination is when a chatbot or other generative AI confidently states something that is false, made up or unsupported. The answer usually sounds fluent and believable, which is what makes it risky. Standards bodies such as the US National Institute of Standards and Technology (NIST) also call this "confabulation."

Why it happens

• Models predict, they don't look up. A large language model generates text by predicting what should come next. That usually produces accurate answers, but it can also produce false ones, especially in long answers and specialist topics.
• Guessing gets rewarded. A 2025 paper by OpenAI-affiliated researchers argues that common benchmarks only score answers right or wrong, so saying "I don't know" earns nothing. Like a student on a multiple-choice test, a model scores higher by guessing.
• Imperfect data. IBM also points to flawed or biased training data and to systems tuned to sound plausible rather than to be true.

Real cases

• Fake court cases (2023). In Mata v. Avianca, lawyers in New York filed a brief citing court decisions that ChatGPT had invented. A federal judge fined them $5,000.
• An airline chatbot (2024). Air Canada's website chatbot gave a customer wrong information about a bereavement discount. A Canadian tribunal held the airline responsible for what its chatbot said.

How to reduce it

• Ground the answers. Techniques like RAG make the model look up real documents before answering and cite them.
• Let models say "I don't know." Researchers propose scoring systems that reward admitting uncertainty instead of guessing.
• Keep humans in the loop. Check facts, numbers and citations, especially for legal, medical or financial decisions.