Class 02 — Why AI Lies Confidently

Date: Thursday, Sept 24, 2026

Objectives

• Describe AI training at a conceptual level (data → patterns → prediction).
• Explain why AI produces confident but incorrect answers (the “bad mirror” and probability over truth).
• Understand why wording, context, and memorized patterns change results.
• Run a basic verification loop on any AI claim (ask → check → cross-check → cite).

Agenda (60 min)

Time	Block
0–5	Warm-up: students show their “suspected AI” examples from homework.
5–20	Mini-lesson: training data, tokens and next-token prediction, probability not truth → hallucinations are structural, not a bug.
20–25	Game: predict the next word in several sentences — feel how prediction works. (First cut if short on time.)
25–40	Stump-the-bot lab: everyone first answers a question themselves, then tests the class tool on it — together on the shared screen. Question source: llm-quiz.com — 30 “seemingly simple problems” from a real research paper, each labeled with the measured AI pass rate (e.g. 49.38% = coin-flip territory for the machine, near 100% for humans). We pick a question the AI historically gets wrong, everyone predicts the tool’s answer, run it, record. Debrief with two numbers: how did the tool do, and did the pass-rate label predict it? (Bonus slot, if time: the viral warm-up — “car wash 50 meters away, walk or drive?” — now famously fixed; seeing a benchmark question get patched is its own lesson.)
40–55	Fact-checking core: source quality; primary/official sources; cross-checking two independent sources; spotting confidence without evidence. Live on Perplexity — researching California ballot measures, then auditing the sources it cites: open each link and check whether the claim is actually on the page.
55–60	Exit ticket: write the verification loop in your own words (ask → check → cross-check → cite).

Key terms

• Token — the unit models predict.
• Hallucination — fluent output not grounded in fact.
• Primary source — where claims actually live.
• The “bad mirror” — the model reflects patterns in human training data rather than reasoning about reality.
• Chain of thought (reasoning steps) — newer models (e.g. Qwen) showing intermediate scratchpad steps to self-correct, which helps but can still get derailed.

Seven-Question connection

Q4 (accuracy) — this class is the deep-dive behind “can I ensure it’s accurate?” The verification loop becomes a required part of every log entry from now on.

What we actually covered (Sept 24)

The record of the second class, from the session notes:

The “Bad Mirror” and probability vs. truth

We explored the fundamental limitation of Large Language Models:

• The “bad mirror” analogy: LLMs don’t look things up or analyze truth in real time; they are a mirror reflecting humanity’s text by predicting probable tokens. When we marvel at fluent output, we are looking at statistical patterns reflected back at us, not human understanding.
• Probability over rules: The model chooses what statistically belongs next. When a puzzle matches a famous pattern, the model defaults to the famous answer even if the specific problem has completely different constraints.

Stump-the-bot lab: Logic puzzle failure cases

We tested problems from the 30-question logic quiz (llm-quiz.com) on our shared screen to watch models struggle with constraints, spatial reasoning, and probability:

• The Trolley Problem inversion: We gave the model a scenario where the runaway trolley was heading down the tracks away from 5 people. Instead of noticing that pulling the lever was unnecessary (or harmful), the AI defaulted to its memorized response (“pull the lever”) because the text was statistically closest to the classic ethical dilemma.
• River Crossing with compartments: The classic wolf, goat, and cabbage puzzle — but the prompt gave the boat 3 secure, separate compartments. Humans instantly see that all three fit in one trip; models often still output the multi-trip classic crossing algorithm.
• Bridge Crossing: A group crossing a rickety bridge at night with one torch — but the text didn’t restrict crossing to two people at a time. Humans realized everyone crosses together at the pace of the slowest person (10 minutes); models defaulted to the standard optimization steps.
• Russian Roulette: Spinning the cylinder randomizes the chamber; not spinning shifts to sequential probability. Models struggled to track state and probability shifts across turns.
• Monty Hall & Birthday Paradox: Models failed to update probabilities when new information was revealed (Monty Hall), and tripped over constraints like twin pairs sharing a birthday (treating twin pairs as a single birthday entity vs. recalculating people).
• Reasoning models & Chain of Thought: We looked at newer models (like Qwen) using internal reasoning steps / scratchpads. While reasoning models attempt to self-correct, they can still hallucinate context or drift off course when generating text over longer stretches.

Fact-checking live & Human vs. AI art

• Fact-checking core on California 2026 ballot measures: We tested an AI search engine (Perplexity) on upcoming California ballot measures. We saw how it pulls web sources and summarizes them, but demonstrated why you must open each citation link — models can misread the reference or summarize the measure incorrectly.
• Human connection in art vs. generated output: We discussed what happens when you prompt an image generator vs. drawing yourself. Students noted that AI lacks personal emotion, intention, and the human spark (“reflection of God” in human creation); it merely predicts probable aesthetics.

Homework / log entry

Find and document one AI mistake this week (any tool, any topic). Screenshot or quote it, and note how you caught it.