Time: about 14 minutes. Watch the video or read the reading first, at least as far as the decision aid.
Unit 1 asked you to sort tasks. This one asks you to choose, and then to notice what the choice cost you.
Part 1 — the decision aid, on one real task (6 minutes)
Take one task from the table you built in Unit 1 — ideally the one you moved into a cheaper class. Work the five steps and write down each answer.
| Step | Your answer |
|---|---|
| 1. Which class is this? | |
| 2. Can I move it? To what? | |
| 3. What will the answer contain that needs checking? | |
| 4. Can I actually do that check, with the time and access I have? | |
| 5. Which is the cheapest kind that leaves me a check I can do? |
Step 3 is where most of the value is, so be concrete. "Sources" is not an answer; "three citations I would have to look up in the library catalogue" is. "A number" is not an answer; "a currency conversion whose date I would have to confirm" is.
Step 4 has only two honest answers, and one of them is uncomfortable. If it is "no", say so and go back to step 2: either supply something that moves the class, or accept that what you will get is a proposal rather than a result.
Part 2 — the same question, two kinds (5 minutes)
Now ask the same question twice: once of a plain generation system, and once of a system that can fetch or be given a source. If you only have access to one kind, simulate the second by pasting in the source yourself. Use a question and a source you are free to send to an outside service: nothing personal, confidential or someone else's unpublished work. Unit 13 explains why.
If you have no AI access, use this supplied pair. Both answers are constructed for this activity: no system produced them, and neither searched. The question: Does this course need prior programming? The source: "What you need before you start" in the Unit 0 reading.
- Answer 1 (nothing attached): Introductory AI courses usually expect some Python, so basic programming is recommended.
- Answer 2 (source supplied): No. The Unit 0 reading assumes no prior programming anywhere, but every learner needs Python 3 installed.
Keep both answers. Record whether each one searched: a product may search without being asked, and then your first answer is not plain generation. Then write three or four sentences on this:
What is in the second answer that I can check, that I could not check in the first?
Note what the question is not. It is not which answer is better written, and it is not which answer is right — you may not know that yet. It is about what each answer gives you to work with.
Two things to look for specifically:
- Does the grounded answer point at a place in the source, or only name the source? A locator is the part that says where to look, such as a page or a clause; "As the policy states" is not one.
- Does the grounded answer say anything the source does not? This is the characteristic failure of the retrieval-backed kind and you may well catch one here on your first try.
Part 3 — price it (3 minutes)
For the task in Part 1, write one line each:
- Money and time: was the more elaborate option meaningfully slower or more expensive for this task? If you do not know, say so — it is the honest answer for most consumer tools, and it is a finding about how visible these costs are.
- Checking: did the extra layer add checking work or remove it?
That second line is the one this unit is for. An extra layer earns its cost when it removes a check you could not otherwise do. It does not earn it when it produces more claims you were never going to verify.
Now take the checkpoints
cp04-what-changes-the-check gives you four observations about an answer and
asks which check each one makes necessary.
cp04-cheapest-adequate is about Part 3's judgement.
cp04-research-mode asks what forty linked citations can and cannot show.
python3 selfcheck.py run cp04-what-changes-the-check
python3 selfcheck.py run cp04-cheapest-adequate
python3 selfcheck.py run cp04-research-mode # or in the browser
Run them in a terminal from The verification lab and self-check, or use the browser self-check.
Optional extensions
Optional. These sit outside the core study time, are not required for the checkpoints, and nothing later in the course depends on them.
Break a chart reading (about 6 minutes). Find a chart whose axis does not start at zero, one with a log scale, or one with values that fall between gridlines, and ask a multimodal system to read a specific value off it. Then read the value yourself. Record what it said, what the value is, and — the useful part — how confident the wording was. This is the fastest way to see why a reference you cannot search directly is a different problem from an unreliable one.
Find a citation that is real and does not support the claim (about 8 minutes). Ask a retrieval-backed system a question in an area you know, and check not whether the sources exist but whether they say what the answer says they say. The failure you are looking for is a real source that does not support the claim made from it; Unit 9, on the routes that include it, names its kinds. This is a preview, and finding one yourself is worth more than being shown one.
Check what a research report left out (about 10 minutes, if you have a research mode). Ask one for a report on a topic you are studying. Then take a source you chose yourself (a review article, a library database search, your reading list) and list three works it names that the report does not, and one the report names that your source does not. Neither list says which is right. Together they show that both are selections, and that the report's citations could not have told you what it left out.
Marking guidance — open this once you have done the activity, to check your own work
Read this after attempting the activity.
Part 1 — what the steps are testing
Step 3 and step 4 are the whole exercise; steps 1, 2 and 5 are bookkeeping around them.
A strong step 3 names artefacts, not categories: which citations, which number, which part of which image or recording. A weak one names a kind of thing, which is the same failure Unit 0's verification sentence had — "I will check the sources" cannot be done, because it does not say what would be opened.
Step 4 has a failure mode worth naming: answering it about the system rather than about yourself. "It is usually reliable for this" is not an answer to "can I do this check". The question is about your time, your access and your competence, and it is the only step that can tell you to change the task rather than the tool. Doing the check is also how you find out whether using the system paid off on this task: whether it saved you time, found what you needed, or did the task better than another way would have. Part 3 asks about part of that: time, money and checking.
If step 4 came out "no" and you did not go back to step 2, that is the thing to correct. A check you cannot do is not a check.
Part 2 — what to look for in the comparison
The intended discovery is that the grounded answer is not better, it is checked in a different way. Three patterns come up:
| What you found | What it means |
|---|---|
| The grounded answer names a source but no place in it | You have provenance, not a locator. Provenance tells you where the answer came from; a locator is what makes the check cheap. Unit 9, on the routes that include it, explains why: a claim with a locator leaves you a check with a clear limit, and one without leaves you a research task. Ask for the page, section or clause. |
| The grounded answer says more than the source does | The characteristic failure of the retrieval-backed kind, and the reason attaching a source moves the check rather than removing it. |
| Both answers agree | Worth nothing on its own. The second answer is not independent evidence for the first — they may share the same source of error. Unit 3 has the mechanism. |
That last row catches people out, because agreement feels like confirmation. It is the same error as re-asking a question and taking the repeated answer as confirmation.
If you used the supplied pair, Answer 2 names its source but no place in it, and its second clause is not in the source: the reading asks for Python 3 only on the Python route.
Part 3 — pricing
"I do not know what it cost" is the honest answer for most consumer tools, where money and energy are invisible at the point of use and time is the only cost you feel. Recording that is a finding, not a gap: a cost you cannot see is a cost you cannot weigh.
The line that matters is the second. The instinct to reach for the most capable available option treats capability as free. It is not free even when it is unpriced, because every extra layer produces another kind of claim, and a claim you do not check is not a benefit.
The checkpoints
cp04-what-changes-the-check matches four observations to the check each makes
necessary. The confusion it targets is treating the presence of a source or a
computed number as the end of the checking rather than the start of it.
cp04-cheapest-adequate tests the judgement in Part 3. The tempting wrong answer
is the one that picks the most capable system because it is available, which is
the reasoning this unit is written against.
cp04-research-mode tests the reading's research modes. The tempting wrong
answers turn five good citations into a verdict on all forty, or on the field;
the right one keeps the two questions apart.
Your private activity record
Browser storage is not a permanent copy
Progress is kept only in this browser, profile and device. Private browsing, clearing site data, removing the profile, a browser reset, storage eviction or device loss can erase it. Keep important answers and contributions separately.
These notes stay in this browser unless you download a backup or activity log.