Spotting AI hallucinations: a practical checklist

A hallucination is when a model states something false with the same calm confidence it uses for things that are true. Not a typo or a hedge, just a wrong fact delivered fluently. That fluency is exactly what disarms us, which is why the problem is worth a whole post. A messy, uncertain human draft invites a second look. A polished paragraph with a citation at the end does not, even when the citation is invented.

I spend a fair amount of my time teaching people to use these tools well, and the one habit that separates the people who get burned from those who don't is knowing where to look. Nothing technical about it. So here is the short version: why this happens, and a checklist you can actually run.

Why models hallucinate (the one-minute version)

It helps to know that this isn't a bug someone forgot to fix. It falls out of how the models are built and graded.

The clearest account I've read comes from Kalai et al. (2025). Their argument is that hallucinations start as ordinary errors in binary classification: if the model can't reliably tell a false statement from a true one during training, some falsehoods survive as confident output through plain statistical pressure. The more pointed half of their case is about evaluation. Models are optimised to be good test-takers, and on most benchmarks a confident guess scores better than "I don't know." We have, in effect, trained them to bluff, because bluffing wins on the scoreboard. OpenAI made the same point in its accessible writeup of that work (OpenAI, 2025): the fix they propose isn't a cleverer model so much as changing how we score the existing ones, so that admitting uncertainty stops being penalised.

Rozear (2026) adds a few practical reasons the problem persists into 2026: training data that contains contradictions and unverified claims, a design bias towards being agreeable and validating (which nudges the model towards overconfidence), and genuine trouble with pragmatics, the contextual and nuanced side of language. None of this is mysterious. It just means the failures cluster in predictable places, which is what makes a checklist possible.

The checklist

Run this against anything you're about to rely on. You won't need every item every time. The first three catch most of it.

1. Find the checkable specifics. Hallucinations concentrate in the parts that look most authoritative: names, dates, numbers, quotes, statutes, citations, API methods. Vague prose is usually safe. A precise "according to a 2024 study, 73 percent of..." is exactly where to slow down. If a claim carries a specific, treat the specific as unverified until you've checked it.

2. Check the sources, don't just admire them. Ask for citations, then confirm three separate things: the source exists, it actually says what the model claims, and the link resolves. Fabricated-but-plausible references are a signature failure. A real-looking DOI or a confident author-and-year is not evidence of anything on its own.

3. Match the question to what the model can know. Anything past the training cutoff, anything private or internal to your organisation, anything hyper-local or very recent is outside what the model reliably knows. Without live retrieval, those answers are reconstructions, not lookups. Knowing the boundary tells you when to distrust a fluent answer by default.

4. Watch for the agreeable answer. If you pushed back, hinted at the answer you wanted, or phrased the question as a leading one, the model will often oblige. Try asking the same thing neutrally, or ask it to make the opposite case. If the position flips under mild pressure, it was never grounded.

5. Notice when it's too tidy. Real evidence is lumpy. An answer where every example lines up perfectly, every number is round, and nothing contradicts the thesis is often a plausible recombination rather than a report of how things are. Tidiness is a yellow flag, not proof, but it's a reliable place to point your scepticism.

6. Make verification cheap, and scale it to the stakes. Cross-check against a primary source, or ask a second model the same question cold and see if the answers agree. For low-stakes work this can be a glance. For anything that goes in front of a client, a patient, a court, or a published paper, verify independently before you trust it, every time. The goal is to make checking so routine that you never have to remember to do it.

7. Ask the model to flag its own uncertainty. It's imperfect, but telling it to mark which claims it's confident about and which it's inferring, or to say plainly when it doesn't know, surfaces the soft spots. Treat the flagged parts as a to-do list for item 2.

The one habit underneath all of it

If you strip the list down, it's a single move: separate the writing from the checking, and never let the writing's confidence stand in for the checking. The model is genuinely good at the first job and genuinely unreliable at the second, so keep that second job yours.

That's also why I'm not gloomy about any of this. The failures land in predictable places, and once the checks are a habit they cost almost nothing. Don't stop at "the AI said so." Go check.

Sources

Kalai, A. T., Nachum, O., Vempala, S. S., & Zhang, E. (2025). Why language models hallucinate. arXiv. https://arxiv.org/abs/2509.04664

OpenAI. (2025, September 5). Why language models hallucinate. https://openai.com/index/why-language-models-hallucinate/

Rozear, H. (2026, January 5). It's 2026. Why are LLMs still hallucinating? Duke University Libraries. https://blogs.library.duke.edu/blog/2026/01/05/its-2026-why-are-llms-still-hallucinating/