Unit 4: Choosing the tool

Reading time: about 12 minutes. Activity: about 14 minutes.

1. What this unit adds

Unit 3 described what a language model does: attention over a context, a distribution over next tokens, and optional layers built around that base — post-training, retrieval, tools. Unit 1 sorted tasks by where the reference lives.

This unit joins them. Deployed systems differ in which of those layers they have, and each layer you add changes what you have to check — not by removing checks, but by moving them somewhere else and adding one of its own.

The practical claim is narrow: choosing a system is choosing a checking job. If you know which one you have taken on, you can pick the system whose checking job you can actually do.

2. Four kinds of system

Not products. Kinds, named by what they add to the base model. A single product may be several of these at once, and may change which it is between one release and the next, so identify what a system is doing on this occasion rather than what its name is.

Plain generation

A base model with post-training and nothing else. Everything in the answer is generated, so a task in Unit 1's recall class stays there: the reference is external and you must go and find it.

There is nothing wrong with this kind. It is the right choice when the class is transformation — you supplied the content, so you already hold the reference — and it is usually the fastest and cheapest option. What it is not is a way of finding things out.

Retrieval-backed

The system searches, or is given documents, and generates an answer from what came back [L3, sec. 2]. This is a genuine difference and it is the most useful single upgrade for the recall class, because it moves the task into Unit 1's grounded answering.

It does not remove the check. It splits it into two questions that are now answerable:

1. Does the source say what the answer says it says? The answer is still generated text about the source, not an extract from it. A summary can stretch a source's qualification, or attach a real citation to a claim the source does not make.
2. Is this the right source? Retrieval returns what matches the query, which is not the same as what settles the question. An outdated page, a draft, a forum post and the authoritative specification can all match well.

The second question is the one people skip, because a citation looks like the end of an argument rather than the start of one.

Whether a system searched is not always your choice. As of November 2026, one provider's documentation says its assistant may search the web without being asked [D4, opening note], so the same product can be plain generation on one occasion and retrieval-backed on the next. Record whether it searched: the checks you need differ.

Research modes. Some assistants offer a mode that turns a question into a research plan, runs many searches one after another, and returns a long report with links to its sources; one provider described its version this way in December 2024 [D5, "Deep Research: Your personal AI research assistant is here to help"]. It is retrieval run in a long loop, and the volume is the cost: a report with forty citations owes forty checks of the two kinds above. It also raises a question no single citation can answer: what was searched, and what was left out? You may be able to revise its plan [D5, "Deep Research: Your personal AI research assistant is here to help"], but which searches it ran, which sources it kept and when it stopped are not something its citations show you. Checking two of its citations tells you that those two are sound; it tells you nothing about whether the report covers the field or only the part that was easiest to find. Its own account of what it searched is generated text like the rest, so before relying on one for an essay or a literature review, compare what it found with a source you chose yourself (a review article, a library database, your reading list), and say what remains uncertain.

When the check is the research. In research of your own there may be nothing to look up at all: at the edge of what is known (Unit 1), the reference does not exist until you make it. Then the check is the research itself, whether that is a derivation, a computation or an experiment. What any of the four kinds of system gives you there is a candidate, a hypothesis to test, and testing it is part of the task, not an extra on top of it.

Tool-calling

The system calls something deterministic — a calculator, a database query, code it wrote and ran, and uses the result.

The check genuinely splits, and the halves are unequal:

• The computation is deterministic and, within its own terms, either right or wrong in the ordinary way. This half is a real gain: arithmetic done by a calculator is arithmetic done by a calculator.
• The decision to call it, with those inputs, for this question is generated. No fixed procedure chose which numbers went in, and none decided what the result means.

So the failure to look for is a correct computation of the wrong question: the right formula on the wrong column, a currency conversion at an unstated date, a query that answers a slightly different question than the one you asked. The result carries the authority of arithmetic and the fallibility of the choice that produced it.

Where the tool is code the system wrote, there is a second failure: a wrong computation. Generated code runs deterministically and can still be wrong, which is what Units 7 and 8 are about.

If the code, its inputs and its output are shown to you, they are evidence you can rerun. If you are only told that a result was computed, you have a claim about a computation. And when a tool reports "all tests pass", open the log and ask whose tests they are.

Unit 8's lab is this idea taken seriously in one direction: run the code, compare against a reference you built yourself.

Multimodal

The system accepts images, audio or video, or produces them.

What changes is that the reference stops being directly searchable. When the source is a document you can find the sentence the answer relies on. When it is a photograph, a chart, or a recording of sound or video, search may need a transcript, OCR (software that reads the text in an image) or an image-search tool, and whatever it locates still has to be checked against the original, by eye or by ear.

Three specific hazards, all worth naming:

• A description of an image or a recording is generated text about it. It can be detailed, written in a confident tone, and about something that is not in the frame or on the recording. A fabricated citation can be looked up in a catalogue and found missing. For an invented detail there is no catalogue: you have to look or listen yourself.
• A chart read by a system is a chart read from pixels. A precise-sounding reading can be wrong when values fall between gridlines, when an axis does not start at zero, or when each step on the scale multiplies the value (a log scale).
• A picture, a recording or a screenshot looks like a record of what happened, and it may have been generated or altered. Treat it as a claim about where it came from: find where it was first published, and what else records the same thing.

Systems that act rather than answer — tool calls with side effects, multi-step loops that carry out tasks — change the question again, because you can throw away a wrong answer, but you cannot undo a wrong action. That is Unit 11's subject and this unit does not anticipate it.

3. A decision aid you can apply in a minute

For a task you actually have, in order:

1. Name the class (Unit 1). Where would the reference come from?
2. If the class is recall, try to move it. Can you supply the source? If yes, do that and use whichever system lets you attach it. This single step removes more checking work than any choice of product.
3. Ask what the answer will contain. Fetched sources? A computed number? A reading of an image or a recording? Each of those is a specific check you are taking on.
4. Ask whether you can do that check. Not "is the system good" — can you, with the time and access you have, carry out the check its output will need? If not, you have chosen the wrong kind, or the wrong task.
5. Take the cheapest kind that leaves you a check you can do. Extra layers are not free, and the next section is about that.

Step 4 is the one that does work. Being able to do the check is what makes the system usable for your task at all, and doing it is also how you find out whether using the system paid off: whether it saved you time, found what you needed, or did the task better than another way would have. A system that produces claims you cannot check is not more useful than one that produces fewer claims you can.

Makers publish scores on standard tests, called benchmarks. A published benchmark score does not answer step 4 either. It means little if the test questions were in the model's training data [E17, p. 10776], [E18, sec. 2.2]. And even a fair score tells you about the test's tasks, not about yours.

4. What it costs to ask

Three costs, and they often move together because they share drivers.

Money. Where a service charges for use, it may price by the amount of text going in and coming out, and by the size of the model; price lists change, so check the current one. A longer conversation can cost more than a short one for the same question, because the conversation's history may be sent with it.

Time. Roughly the same drivers, plus the extra steps: retrieval adds a search, tool-calling adds an execution, and a system that generates intermediate text before answering generates more text.

Energy. Real, and driven by the same quantities: more computation for longer inputs, larger models and more steps.

A figure this course will not print

You will have seen numbers for what a single query costs in energy or carbon. This course does not give one, and the reason is the course's own subject. Every page and narration script was searched for a number with an energy or carbon unit beside it, and there is none.

A defensible figure would have to name the model, the hardware, the length of the query, whether the cost of training the model is shared across queries, and the electricity supply, and it would be out of date within months. Printing a number without those conditions would be exactly the confident-and-unsupported claim this course trains you to catch.

So instead: the shape is reliable even when the numbers are not. Other things equal, more computation generally uses more resources. Money and time also depend on pricing, on hardware, on caching (reusing stored results) and on how the work is executed. If you need an actual figure, that is a Unit 1 recall task, and it needs a source that states its conditions and its date.

When the cheap option is the right one

Often. The instinct to reach for the most capable available system treats capability as free, and it is not — in money, in time, and in checking work, because a more elaborate system produces more kinds of claim to check.

The cheap option is right when the class is transformation or grounded answering and the check is a comparison you will do anyway; when the task is a draft you will rewrite; and whenever the expensive option's extra output is something you would not check.

The expensive option earns its cost when the extra layer removes a check you could not otherwise do — retrieval on a question you cannot research yourself, a tool call on a computation you cannot perform, and when you will actually use the check it makes possible.

5. What this unit is not saying

It is not saying retrieval-backed and tool-calling systems are more trustworthy. They are checked in different ways. Each removes one kind of uncertainty and adds a different one.

It is not a recommendation of any product. Named products date, change what they are between releases, and are not the level at which this reasoning works.

Summary

• Systems differ by which layers they add to a base model, and each layer moves the check rather than removing it.
• Plain generation leaves the whole check with you; retrieval aims it at a source; tool-calling splits it into a computation and the choice to make it; multimodal puts the reference somewhere you cannot search directly.
• A research mode is retrieval in a long loop: every citation still needs its check, and none of them says what was left out. Compare with a source you chose.
• In research of your own the check can be the research itself: a derivation, a computation or an experiment. The output is a hypothesis to test.
• A picture or a recording is a claim about where it came from, not a record.
• Choose by asking what check the output will need and whether you can do it, not by which system is most capable. Doing the check is also how you find out whether using the system paid off.
• Other things equal, more computation uses more resources: longer inputs and outputs, larger models, more steps. Money and time also depend on pricing, caching and hardware. The shape is stable; the numbers are not, and this course declines to print one it cannot defend.
• The cheapest kind that leaves you a check you can carry out is usually the right one.

Then answer cp04-what-changes-the-check, cp04-cheapest-adequate and cp04-research-mode at the foot of this page.