Updated June 2026

“Do models hallucinate?” is the wrong question; the answer is yes and always will be at some rate. The useful question is which outputs deserve suspicion, because reliability isn’t uniform. It follows a pattern you can learn.

The gradient

Most reliable: transforming text the model was just shown. Summarize this document, rewrite this paragraph, extract the dates from this email. The source is in the context window; the model is doing transcription with judgment. Errors happen but they’re rare and usually small.

Reliable: general knowledge and reasoning. Explaining concepts, comparing approaches, walking through logic. Broad, heavily-documented knowledge is solid ground.

Shaky: specific facts from memory. Version numbers, API signatures, configuration flags, dates, statistics, quotes, names of minor things. This is where models confabulate fluently: the answer arrives in perfect confident prose whether it’s right or wrong. The model has no internal signal that distinguishes “I know this” from “this is the most plausible-sounding completion.”

Least reliable: citations and references from memory. Papers, URLs, court cases, library functions. These have the exact shape of real ones, and checking them is non-negotiable. The same goes for anything where being plausible and being correct diverge sharply.

The tells

Confidence is not a tell; everything is delivered confidently. The actual signals: suspiciously specific details nobody would memorize (an exact percentage, a precise date for a minor event), perfectly convenient answers that match what you hoped, and specifics that shift when you ask again in a fresh session. Inconsistency across samples is one of the few honest signals available.

What to do about it, in order of leverage

  1. Ground instead of recall. The single biggest fix: put the source in the context. Paste the docs, fetch the page, attach the file. You move the task from “shaky” to “most reliable” on the gradient above. This is the entire reason RAG and web-search tools exist.
  2. Verify where verification is cheap. Code runs or it doesn’t. Links resolve or they don’t. Commands can be tried. A huge fraction of model output is mechanically checkable, and an agent with tools can be told to check its own claims (“run the tests before reporting done”).
  3. Ask for uncertainty explicitly. “Say so if you’re not sure” genuinely helps, because the default is trained-in helpfulness, not trained-in calibration. It won’t catch everything, but it surfaces the cases the model itself can flag.
  4. Match scrutiny to stakes. A wrong fact in a brainstorm costs nothing. A wrong fact in a legal filing, a medical decision, or a production config costs plenty. Spend your verification budget where errors are expensive and silent.

The working posture: treat model output like a smart colleague’s first draft from memory. For prose and reasoning, edit it. For any load-bearing specific, check it.