patrickz.aiLet’s talk ↗

Start with AI / Guide

Check an AI answer for made-up facts in 15 minutes

See why a fluent answer can still be wrong, then check an AI’s claims outside the chat.

About 15 minutesBeginnerBeginner path · step 2 of 6Read free · No sign-up

Before you start

Any chat assistant, a web browser for checking, and a topic you know well. No coding required.

Why this lesson exists

Patrick’s AI-literacy anime uses a dramatic story to explain next-token prediction. A language model builds a response in pieces based on context; fluent delivery is not independent evidence that a statement is true. When a model states something false with confidence, people call it a hallucination.

The idea this guide practices: How AI actually answers (and why it sounds sure when it’s wrong). Prefer a story? Read the comic The Infinite Confidence Tournament.

Do the exercise

  1. Observe a continuation

    Ask the assistant to continue “The bank was…” in a river scene and then in a finance scene. Compare the words that become plausible when context changes.

  2. Ask for checkable facts

    Ask: “Recommend three books about [a hobby or topic you know well], with exact title, author and year.” Save the reply exactly as given.

  3. Check every item outside the chat

    Look up each title and author in a library catalog or bookstore search, not in the same chat. Mark each one found as described, found with different details, or not found. A tidy, confident list is not evidence.

  4. Add a rule and compare

    In a new chat, repeat the request with: “Only list books you are confident exist; write not sure instead of guessing.” Check the new list the same way. The rule can reduce errors; it cannot replace your check.

Try it now with a ready-made sample

Everything is filled in. Copy it, paste it into your assistant and compare the answer with the checks below.

Recommend three books about growing tomatoes on a balcony. For each give the exact title, author and year of publication.

Adapt it to your own task

Replace the bracketed parts with your own details.

Recommend three books about [topic you know well]. For each give the exact title, author and year of publication.

More prompts to adapt ↗

What this looks like

A reply can pair a real author with a book they never wrote, in perfect formatting. If any item is “not found” or has the wrong year, you have seen the lesson: the model predicts plausible text, and checking it is your job. If all three check out, your check is still what made them trustworthy; try a narrower topic next time.

A good answer includes

  • Each book marked found, different or not found, based on a catalog search
  • One sentence on what the “not sure” rule changed

Check your result

Use evidence from your output. A confident explanation from the AI is not enough.

If it isn’t working

If the assistant searches the web and shows links, open each one and confirm it describes that exact book; a link is not a check until you read it. If you cannot confirm a book exists, mark it unverified instead of trusting the chat.

Next in the beginner path: Write an AI context pack so answers fit your situation →

Where this came from

Original LinkedIn post ↗. The practice lesson is an adaptation, not a verbatim transcript. About the sources.

Prepared September 2026. Tools and interfaces change; use current official setup instructions. Session lengths are estimates.

KEEP GOING

Your next useful step

Browse all 57 guides ↗

THE IDEA BEHIND THIS GUIDE

Read the story, then keep practicing