Last fact-check: May 25, 2026
A fully verified AI conversation is a contradiction in terms. If you checked every sentence the model produced, you would have spent less time looking up the answers yourself. The point of using AI is to save effort, and verification costs effort. So the goal can’t be to verify everything. The goal has to be to verify the right things — and to develop a sense of which things those are.
This is the third article in Tygart Media’s free AI Literacy curriculum. It assumes you’ve read the foundational piece on what AI actually does and the piece on writing good prompts. Verification is the third leg of basic AI literacy. Without it, the other two are useless. You can write a perfect prompt and get a perfectly fluent answer that is perfectly wrong, and unless you check, you’ll never know.
The asymmetry that makes verification necessary
Start with the underlying problem. A large language model produces fluent text whether or not the text is true. The fluency does not vary with the truthfulness. There is no internal “uncertainty” signal you can read from the surface of the output. A confident answer and an answer the model just made up look identical.
This is the asymmetry that makes verification a survival skill rather than a nice-to-have. If wrong answers looked obviously wrong, you wouldn’t need to check. They don’t. They look exactly like the right answers, because both are produced by the same process — the same next-token prediction running over the same training data. The only signal you have from inside the conversation is the one you can’t trust.
The signal you can trust lives outside the conversation. That’s where verification has to happen.
Triage: what actually needs checking
The first move is deciding what’s worth verifying at all. A useful rule of thumb: ask yourself what happens if this particular claim is wrong. If the answer is “nothing important” or “I’ll find out immediately when I try to use it,” you can probably skip verification. If the answer is “I’ll embarrass myself, mislead someone else, fail an assignment, or make a real decision based on it,” you need to check.
Some claims that almost always need checking:
- Any specific number, date, or statistic
- Any quotation attributed to a specific person
- Any citation, source, paper, book, or court case
- Any name of a real person, organization, or law
- Any claim about current state — who holds a role, what the law currently is, what something currently costs
- Any technical claim where being wrong costs you (medical, legal, financial, structural, safety-related)
- Any claim about a specific product feature, version, or capability
Some claims that usually don’t need verification at the same level:
- General explanations of well-established concepts (the model has seen these explained many times and the average is reliable)
- Suggestions, ideas, brainstorming — these don’t have to be “true,” they have to be useful
- Tone, framing, style, structure suggestions
- Code that you’re about to run anyway — the runtime is its own verification step
- Edits and rewrites of text you provided — you can see what changed
The dividing line isn’t whether something could be wrong. Everything could be wrong. The dividing line is what wrongness would cost you.
The five failure modes worth knowing by name
Not all AI errors are the same kind of error. Knowing the common failure modes makes you faster at spotting them.
1. Fabricated specifics. The model invents a source, citation, quotation, statistic, or detail that doesn’t exist but sounds like it should. This is the classic hallucination. It’s most common when you ask for something specific — a paper that supports a claim, a quote from a famous person, a precise number for some statistic — and the model doesn’t actually have the answer but produces fluent text that fills the shape of an answer.
2. Outdated specifics. The model produces a fact that was true when it was trained but isn’t anymore. Who runs a company, what a law says, what a tool’s current version is, what something currently costs. The model has no awareness that its training data has a cutoff. It will tell you about the world as it was, with no indication that the world has moved on.
3. Plausibility traps. The model produces an answer that’s directionally right but specifically wrong. Right author, wrong book. Right concept, wrong year. Right principle, wrong formula. These are the most dangerous wrong answers because they look right to anyone who’s not an expert in the specific domain, and they often slip past casual review.
4. Confident wrongness on edge cases. The model is excellent on common cases and unreliable on edge cases. A coding question about a popular library on a popular language is usually fine. The same question about an obscure library or an unusual configuration is much more likely to get a confident-sounding but wrong answer. The output looks the same. The reliability is not.
5. Sycophantic agreement. Covered in the foundational article. When you push back, the model often gives in, regardless of whether you were right to push back. If you got an answer, then said “are you sure?”, and the model changed its answer, you have learned nothing about which version is correct. You’ve only learned that the model is sensitive to your skepticism.
Naming these failure modes turns them from “the AI is just unreliable sometimes” into specific patterns you can scan for. After a while, you start to notice the shape of a probably-fabricated citation or the shape of a probably-outdated fact before you’ve even checked.
Verification techniques, in order of effort
Verification doesn’t have to be heavy. Most of the time, light verification is enough. A practical hierarchy, from cheapest to most expensive:
Cross-reference within the same chat. Sometimes the simplest check is asking the model to verify itself. “What’s your source for that?” or “Is that current as of when you were trained?” can surface uncertainty the original answer hid. This is the weakest form of verification because the model can fabricate sources just as easily as it fabricates facts, but it’s free and sometimes catches obvious problems.
One quick search. Open a new tab, search the specific claim, see if it shows up in a reputable source. This is the workhorse verification technique. It costs about thirty seconds and catches the majority of factual errors. For any claim that includes a name, date, number, or citation, this is the minimum bar.
Check the primary source directly. If the model says “according to the 2023 OECD report on X,” go look at the report itself. Don’t trust the model’s summary. The summary may be accurate. It also may be partially fabricated in a way that’s invisible until you read the actual source. This is what real research looks like.
Independent expert check. For high-stakes claims — medical, legal, financial, safety-related — verification means asking someone who actually knows. The AI’s answer is a starting hypothesis. The expert’s answer is the answer.
The two-model check. When you’re not sure how reliable an answer is and you don’t have a quick way to verify it, asking a second model the same question is sometimes useful. Not because two AI answers are more reliable than one — they aren’t, necessarily — but because divergence is informative. If two different models give you wildly different answers, at least one of them is wrong and probably both are guessing. Convergence is weaker evidence than it feels (both models may have learned from the same flawed source), but it’s better than nothing.
The specific case of citations
Citations deserve their own section because they are the most common, most dangerous, and most widely encountered hallucination. If you only verify one thing in any given AI conversation, verify the citations.
Models fabricate citations with extraordinary fluency. The author name will sound real. The journal will sound real. The page numbers will be plausible. The DOI may even follow the right format. None of it has to be true, because none of it was checked.
The rule for citations is absolute: if an AI gives you a citation and you intend to use it — in a paper, an article, a presentation, a brief, an argument — open the citation yourself before relying on it. Search for the paper. Click through to the journal. Confirm the author wrote it. Confirm it says what the model said it says. If you can’t find it in thirty seconds, assume it doesn’t exist.
This is not paranoia. This is the consequence of using a tool that produces plausible text. The plausibility of a citation is not evidence of its existence. The only evidence of its existence is finding it.
Spotting fabrication before you verify
Over time, you start to notice patterns that correlate with fabrication. They aren’t proof, but they’re signals worth heeding:
- Specific numbers without specific sources. “Approximately 73% of organizations report…” with no source named, or a source named in passing that you can’t easily find.
- Quotes that sound too pat. Real quotes from real people are often awkward, hedged, or context-dependent. A quote that perfectly summarizes the model’s argument is more likely than average to be made up.
- Citations with suspiciously round numbers. Real journal articles don’t usually start on page 100 exactly. Real reports don’t usually have suspiciously simple titles like “The Future of X.”
- Confident statements about recent events. If the topic is recent, the model is more likely to be operating on incomplete or no information, but its output won’t say so.
- Combined claims. “X said Y about Z in 2019.” Three things to check. Each one might be true individually and the combination still wrong.
None of these are conclusive. Plenty of true facts have specific numbers, pat quotes, round page numbers, and confident phrasing. But each is a signal that the verification check is worth running.
What to do when you find a wrong answer
When verification reveals that the model got something wrong, the instinct is to argue with it. “That’s incorrect — the actual figure is X.” The model will then apologize and produce a corrected version. This sometimes resolves the issue. It often doesn’t.
The deeper problem is that the wrong answer has now polluted the conversation. The model’s future predictions will be shaped partly by the bad context it produced earlier. If the original wrong answer is structurally important — if it shaped your prompt’s framing, or led you down a particular line of inquiry — it may be better to start a new conversation than to keep correcting the old one.
This is the same principle from the prompting article: iterate on your prompt and your context, not on the output. Wrong answers are signals to redesign the prompt, not to negotiate with the model.
The professional habit
People who use AI well in professional or academic contexts develop a small set of unconscious habits. Worth naming them, because they’re learnable:
- They mentally tag claims as “verified” or “unverified” as they read the model’s output, without slowing down to verify everything in real time.
- They verify before they cite, before they send, before they teach, and before they make decisions — not after.
- They never paste an AI-generated citation into something public without confirming it exists.
- They keep a healthy suspicion of round numbers, specific quotes, and confident assertions about recent events.
- They treat AI output as a starting point, not an ending point. The model is the first draft of the answer, not the answer.
None of this is exotic. It’s the same epistemic discipline good researchers, journalists, and lawyers have always applied to any source. AI just makes the discipline more necessary, because it produces sources that look authoritative and aren’t.
The closing point
The CSU rollout that motivates this curriculum gave 470,000 students a tool that produces fluent text, with no training in how to tell when the fluent text is true. That is the literacy gap, restated. Closing it does not require students to become AI researchers. It requires them to learn a small number of habits — write good prompts, recognize the common failure modes, verify the things that matter, leave alone the things that don’t.
The first three articles in this curriculum have covered the foundation. What the model does. How to ask it for what you want. How to check whether what it gave you is true. Everything else in the sprint sits on top of these three. If you teach a student nothing else about AI, teach them these three.
About this knowledge node: This is a cluster article in Tygart Media’s AI Literacy content sprint. It’s licensed for use in any classroom, training program, custom GPT, or Claude Project as long as attribution is maintained. The pillar article that introduces the sprint is here. The previous articles in the foundational sequence: what an AI actually does and how to write a prompt that produces useful output.