Unit 9: Grounding and citation checking

Reading time: about 17 minutes. Activity: about 36 minutes.

1. The second worked case

Units 6, 7 and 8 work through one case in full: you build a reference by hand, ask for a routine, compare value by value, and stop at the first divergence. That is the method. It works there because the case has a definite answer. Those units are on the routes with the mathematics. If your route skipped them, this unit is where you meet the method, on a case without a definite answer.

Most of what you will actually check is not like that. There is no oracle (a source of correct answers you can trust independently of the thing being checked) for "does this source support this claim". So this unit runs the same method on a case with no oracle, and the point is that the method survives intact:

This is not a lighter version of the lab. Building the reference is still the work, the comparison is still item by item, and the scoped statement at the end is still the deliverable. What changes is that the last column's judgements are yours, because no program in this course makes them.

2. What a citation from a generative system establishes

By itself: nothing.

A citation produced alongside a claim is another piece of generated text. It sits in the same output, produced the same way, with no more evidence attached than the sentence it follows. Unit 3 has the mechanism; nothing in generation consults a record of what is true, and attaching a bibliography does not change that. A system may search before it answers and attach links to what it retrieved; as of November 2026, one provider's documentation says its assistant may search the web without being asked [D4, opening note]. That plausibly makes the first two questions below pass more often, since the work and the page are ones the system actually fetched. It leaves the third untouched: the sentence that connects the claim to the source is still generated.

But a citation is the most useful sentence in the output, for a reason that has nothing to do with its reliability: it is the one part that is cheap to prove wrong, if it is wrong. An unbounded claim leaves you a research task. A claim with a locator leaves you a bounded one — you know which source to open and where to look. How long that then takes depends on the source: a page you can reach settles it in a minute, a paywalled or dense one does not. Asking for citations is worth doing not because they are evidence but because they are testable.

A citation with a locator is half of a certificate (Unit 0): something that lets you check the claim without trusting whoever made it. The whole certificate is the quotation with its locator, the exact sentence the claim rests on and where it is. Then the check is to open that page and read one sentence, which costs a fraction of what producing the claim did. So ask for the quotation, not just the reference. And notice what a checked quotation certifies: that the source says this, not that the source is right.

Whether the source deserves to be relied on is a separate check, and the source itself is a poor place to make it. Wineburg and McGrew describe how professional fact checkers judged an unfamiliar website: they left it early and read what other sources said about who stood behind it, which they call lateral reading, rather than staying within the site itself to judge it, which they call vertical reading [E19, abstract]. The same move works on a cited report or an unfamiliar journal: before you weigh what it says, find out from somewhere else who published it and what they are known for.

Pictures, recordings and screenshots

Everything here applies when the evidence is an image, a recording or a screenshot rather than a sentence, with one difference: a picture looks like a record of what happened, and it may not be one. Generated and altered images now get past people trained to judge them. In a 2026 study, reviewers with PhD-level training were shown research images altered by a generative model to change the result, and identified them no better than chance; the best automated detector tested did better, but was far from reliable [V8, "Results"]. An earlier study found free AI-image detectors unreliable on laboratory images [V7, "Discussion"].

So treat an image as a claim about where it came from, and check it the way this unit checks a citation: find where it was first published, and what else records the same thing. Looking right proves nothing, and neither does a missing provenance label (a label saying where a file came from or how it was made): its absence shows nothing about how a file was made (Unit 14). If an image, a recording or a voice message asks you to do something that matters, confirm it through a channel you found yourself, not one the message supplies.

Tracing one takes four steps, and they are lateral reading applied to a picture:

1. Search by the image itself. A search engine's search-by-image finds other copies and earlier ones. Try more than one engine; each finds different copies. The earliest copy with a named source is a lead, not yet the origin.
2. Read what came with it, as claims. The caption, the credit, the date and place, and any file details or content credential (a record attached to the file saying how it was made; Unit 14). Each says where the picture came from; none proves it, and all can be stripped or edited.
3. Find an independent record of the same thing. Another photograph from another source, a report from someone who was there, the original publication. For a figure in research, that is the paper itself and, where it is published, the data behind it.
4. Write down what you could and could not trace. "First published by X on this date, matching Y's account" is a finding. So is "no copy earlier than this post, and no independent record": that makes the picture untraced, which is not the same as generated.

None of the six is new

Every failure mode below occurs in work written by people: quotations credited to someone who never said them, textbooks repeating a figure that was superseded years ago, a number sharpened each time it is retold, a press release turning an association into a cause, page references copied from someone else's bibliography. An unchecked citation establishes nothing, whoever wrote it. What generation changes is volume: a referenced summary with eight claims takes seconds to produce and still takes an afternoon to check, and it looks just as finished either way.

3. Three questions, in order

A citation has to survive three questions, and they get very unequal attention.

1. Does the work exist? Author, title, year, venue. This is the cheapest question, and on its own it settles the least.
2. Does the locator exist? The page, section, clause or figure. A real work with an invented page number is possible and is invisible unless you look.
3. Does the source support this claim, as stated, at this strength? A citation can pass the first two and fail here, and this is the one that costs actual reading.

Question 3 is the whole exercise. A summary can cite a real paper at a real page that genuinely discusses the topic, and still misstate what it found, and every surface signal, including the correctness of the first two answers, points the wrong way.

Question 1 is cheap, and it cannot be assumed to have been asked. There is now direct evidence: references to works that do not exist have been found in papers that passed peer review and were published. One team scanned 4,841 of the 5,290 papers accepted at a large machine-learning conference in 2025. They found 100 fabricated citations in 53 papers; two thirds of those citations pointed to works with no real counterpart at all [L10, secs. 2.1-2.2, 3.1]. An audit of about 2.5 million biomedical papers found at least one fabricated reference in about one paper in 2,828 from 2023, one in 458 from 2025, and one in 277 from the first weeks of 2026 [L11]. Those figures measure what reached print, not how often a system invents a reference, and both studies used automated tools to find the candidates. What they show is that publication does not guarantee that a cited work exists. Ask question 1 every time; a library catalogue answers it in a minute.

4. Six citation-failure modes used in this exercise

A closed vocabulary for this case, used with no synonyms, exactly as the code failure modes work in the lab. Closed for this exercise, not exhaustive of reality: a nonexistent work, a selective quotation, a compound claim only half supported, a retraction, and several sources that turn out to share one origin are all real failures outside these six.

These are primary diagnoses, not exclusive descriptions

The six sit at different layers. fabricated-locator is about the pointer, stale-version about which edition, misattributed about which work, and the last three about what the passage actually supports. A real claim can fail at several layers at once — a fabricated page in a superseded edition making an unsupported inference is one claim with three faults, and no wording of the definitions will make the categories exclusive in every case.

So you are naming the primary fault, and there is an order for deciding it. Work down; the first one that applies is the diagnosis.

1. Does the locator exist? No → fabricated-locator.
2. Is an explicitly superseded statement being presented as current? Yes → stale-version.
3. Is the claimed finding absent from the cited work, but supported by another identified source? Yes → misattributed.
4. Does the source support the same claim, population, direction and scope, only less precisely? Yes → over-specified.
5. Does the source support a weaker claim, with the summary adding causation, generality, certainty or scope? Yes → unsupported-inference.
6. Is the cited passage the right place to check, and does it report a different value, population, direction or result? Yes → right-source-wrong-claim.

The order is not arbitrary. It runs from the cheapest check to the most expensive: whether a page exists, then which edition, then which work, and only then what the passage means. That is also the order in which a fault makes the later questions moot — there is no point asking what a passage supports when the passage does not exist.

Two distinctions are worth pinning down, because they are the ones people run together even with the order in front of them.

over-specified against unsupported-inference. Both are "the source says less than the summary", and the difference is what kind of less. Over- specification keeps the claim, the population and the scope and sharpens a quantity — an approximate magnitude becomes an exact figure, "in the 1990s" becomes 1994. An unsupported inference changes the logical force — from association to cause, from some to all, from one setting to any setting.

misattributed against right-source-wrong-claim. Both are "wrong source", and the difference is whether the cited work contains the finding at all. In misattribution it does not: the finding is real and belongs to a different work, so correcting the citation repairs the claim. In right-source-wrong-claim the cited passage is exactly where you would look, and it reports something else, so no citation fixes it, because the claim itself is wrong.

Note what that distinction is not. It is not about whether the source is "relevant". Relevance is a matter of degree and a book can be squarely on a subject without containing a particular result. The operational question is narrower and answerable: open the cited work, and see whether the finding is in it.

These are not detected by a program here

The lab's check.py names b1-sign by re-running a candidate against an oracle. Nothing equivalent exists here. This lab does not automate these classifications. Automated tools may assist with parts of them, such as finding a quotation, a locator or a version, but their scope and errors need checking. Whether a source supports a claim is a judgement about meaning, and a checker that claimed to settle it outright would be committing exactly the overreach this course trains you to catch.

So the discipline is not automation. It is the closed list. Having to choose between unsupported-inference and over-specified forces you to say what kind of wrong you found, and "the citation was bad" does not.

5. Building the reference

This is the step people skip. On the routes with the mathematics it is the hand derivation of Unit 6, done before any generated output is seen; here it is reading the sources before you read the summary.

The reference is what the sources actually say, in your words, written down before you compare. One row per locator the summary uses, plus a row for any passage that changes how another source may be used:

The third column is the one people leave out, and it is where two of the six failure modes are caught. A passage saying "this supersedes the targets in the 2019 review" supports no claim of its own and decides whether another source may still be cited. Passages such as "we could not separate these two effects" or "across the six authorities studied" belong in this column too: they limit what a claim may say.

The middle column is the reference. It is the counterpart of the value you computed by hand, and it has the same property: you produced it, so comparing against it means something.

Record what you could not retrieve. A source behind a paywall, a broken link, an edition you cannot get — those are findings, not gaps in your work. "Could not retrieve" is a legitimate row and it changes what you can claim at the end.

6. Comparing, and stopping at the first divergence

Take the claims in order. For each, ask question 3: does the passage support this, as stated, at this strength?

Stop at the first claim that fails, and name its code.

In the activity's specimen, which is invented for the exercise, set each claim beside what its cited source says at that place, and record whether the source supports it. Claim 1 says the kerbside scheme began in 2011; Source A, section 1.1, says the scheme began in 2011. The same fact at the same strength, so it is supported, and you go on. Claim 2 says participation increased by 19.6 percentage points between 2012 and 2018; Source B, p. 94, says approximately 20. That is the first divergence, and its code is over-specified: same population, same period, same direction and scope, and only the precision is invented.

The reason is the one Unit 8 gives for code, where the first index that disagrees is a place to start debugging and says nothing about the indices after it. It is worth stating here because it feels wrong: later claims may look fine, and their looking fine tells you very little once you know that the summary is capable of producing an unsupported claim in this register. The first divergence is a precise place to start, not a proof about everything after it.

What it is not is a licence to stop working. You stop the diagnosis there; you still check the rest before you rely on any of it.

7. What you can say at the end

The deliverable is a scoped statement, exactly as it is in Unit 17.

Not "the summary is accurate", which nothing you did establishes. Something like:

Of eight claims, I retrieved sources for six. Five are supported as stated. One cites a section that does not exist in that document (fabricated-locator); I found the same claim supported at section 2.1 of the same work, so the claim stands and the locator does not. Two claims I could not check because I could not retrieve the source.

Every part of that is defensible: it says what you checked, what you found, what you could not do, and what follows. Compare it with "I verified the references", which says none of those things and cannot be interrogated.

8. What this unit is not saying

It is not saying generated citations are usually wrong. It makes no claim about how often, because this course has no measurement of that and neither should you. The claim is about what a citation establishes before you check it, which is nothing, and how cheaply you can change that.

It is not saying retrieval-backed systems remove the problem. They move it, as Unit 4 said. A fetched source is a real source, but you still have to check whether it says what the answer claims.

Summary

• A citation from a generative system is generated text, and establishes nothing by itself, but it is the cheapest part of an output to prove wrong, which is why asking for one is worth doing.
• Whether a source deserves trust is checked by leaving it: lateral reading, finding out from other sources who stands behind it.
• Three questions in order: does the work exist, does the locator exist, does the source support this claim as stated at this strength. The first is the cheapest, and fabricated references in published papers show that publication does not answer it; the third is the one that needs actual reading.
• An image, a recording or a screenshot is checked like a citation: where was it first published, and what else records the same thing.
• Six named failure modes, a closed list, assigned by you rather than by a program, because there is no oracle for what a source supports.
• The method is the one Unit 8 runs on code, against a reference a program can compute: build the reference first, compare item by item, stop at the first divergence, name the fault, and end with a scoped statement rather than a verdict.

Then answer cp09-diagnose-source, cp09-scope-the-claim-source and cp09-image-origin at the foot of this page.