Unit 11: Agents and automated actions

Reading time: about 12 minutes. Activity: about 14 minutes.

What changes

Everything so far has been about an answer: something you read, check, and then either use or discard. Discarding is cheap, and it is the reason the whole method works. You can take as long as you like over a claim, because until you act on it, nothing has happened.

An agent is a system that acts. It calls tools in a loop — plan, act, look at the result, act again, and some of those actions change something outside the conversation. A file is written. A message is sent. A row is deleted. A booking is made.

The method does not change. What changes is one thing, and everything in this unit follows from it:

An action can change the world before you have looked at it.

Some actions are cheap to take back — a draft, a preview, a file in your own bin. Some are not, and a few cannot be taken back at all. The next section separates those, because the difference decides how much checking has to happen before rather than after.

Checking after the fact is still worth doing, because you need to know what happened. But it is no longer verification in the sense the rest of this module uses: it cannot prevent the error, only describe it, and what it leaves you is a remedy rather than a correction.

The loop, and where you are in it

A single-shot answer has one place a check can go: after the output, before you use it. An agent run has many, and by default you are in none of them.

A typical run:

1. The system produces a plan. This is generated text.
2. It selects a tool and arguments. This selection is generated.
3. The tool executes, under its own semantics and against the state of the world at that moment.
4. A result is recorded.
5. The system reads the result and decides what to do next. Generated.
6. Repeat, possibly many times.
7. It reports what it did. This report is generated text about the run.

Step 3 is not the safe step. It is tempting to say that the generated parts are the risky ones and the tool merely does as it is told, but that is only true of a tool with no outside world in it — a calculator, say. Most tools have one. A search returns what the index holds today. A file operation depends on what is on disk, on permissions, and on what else is writing at the same time. A network call can time out, half-finish, or succeed after the system has given up waiting. A tool can carry its own bug. And a tool can do exactly what it was told while doing the wrong thing, because what it was told came from step 2.

So there are three distinct places a run goes wrong, and they need different checks:

ok is a claim about a call, not about the world. A write that returns success may have written to the wrong path; a delete that returns success may have matched nothing. The check for step 4 is to look at what is actually there afterwards.

This is Unit 4's tool-calling split, run repeatedly, with the consequences compounding: an error at step 2 produces a result at step 3 that the system then reasons from at step 5.

Reversibility is the axis

Unit 3 sorted outputs by what kind of check they need. For actions, that is the wrong first question. The first question is:

If this is wrong, can it be undone, by whom, and at what cost?

Four rough bands, and they are not about how likely an error is:

"Undo" is doing a lot of work in that table, and it is worth separating four things it can mean, because they cost different amounts:

• Undo — the action is reversed and no trace of it remains that matters. A file restored from your own bin.
• Compensating action — the original stands and a second action offsets it. A refund does not unsend a payment; a correction does not unsend a message. This is the usual remedy in the third band, and it is why that band is not the same as the fourth.
• Recovery — the state is rebuilt from something kept for the purpose: a backup, a version history, a transaction log. Only available if someone arranged it beforehand.
• Containment — the action cannot be undone or offset, and what is left is limiting what follows: revoking a key, notifying the people affected.

A message sent to the wrong person is not undoable and is usually correctable. A disclosure of someone else's personal data is neither, and containment is all there is. Ask which of the four you would actually be reaching for.

Note what is missing from that table: any assessment of how reliable the system is. That is deliberate. Reliability changes how often you will need the remedy; it does not change what the remedy costs. Repeat a bottom-row action often enough with a system that is usually right, and the question stops being whether an unrecoverable error is likely on this run and becomes whether you are willing to carry one at all. How soon depends on the error rate and on whether the failures are independent, neither of which you know, which is the point. You are choosing an exposure, not calculating a date.

What a trace does and does not tell you

Agent products commonly show you a trace: the plan, the tool calls, the results, the summary. Read it — it is far more than a single-shot answer gives you.

Then keep two distinctions.

The tool calls and their results are records — if what you are reading is the system's own log. A call was made with these arguments and returned this. That is an artefact of what happened, and it is the most checkable thing in the run.

That "if" matters, because three different things can look identical on screen:

The third is the one to watch, because it is written in the same voice as the first. If you cannot tell which you have, that is worth finding out before you rely on it: a product's documentation says whether it shows you the log or a summary of it.

The plan and the summary are generated text about the run. They can be accurate. They are not a transcript, and a summary can describe a run that did not happen that way, not by deception, which would require intent, but because the summary is produced by the same process as everything else and nothing checks it against the log.

So: check the tool calls, not the narration. If a report says "I checked the existing entries before adding the new one", the question is whether a call that would have done that appears in the trace.

And note what is often missing. Intermediate steps may be summarised or hidden; a long run may show you a condensed version. The absence of a step from a displayed trace is not evidence that it did not happen, and its presence in a summary is not evidence that it did.

A worked trace, with the checkable points marked

The task: "Find the duplicate entries in my references file and remove them."

[1] PLAN    "I will read the file, identify duplicates by title,
             and delete the later occurrence of each."   <- generated
[2] CALL    read_file("references.bib")                  <- checkable
[3] RESULT  412 lines returned                           <- record
[4] PLAN    "I found 6 duplicate titles."                <- generated
[5] CALL    write_file("references.bib", 380 lines)      <- checkable
[6] RESULT  ok                                           <- record
[7] REPORT  "Removed 6 duplicate entries."               <- generated

Six things are worth checking, and they are not evenly important.

• [2] the file. Right file, right path. Cheap, and the most consequential early check.
• [4] the six. "I found 6 duplicates" is a claim with no evidence attached. Which six? A duplicate identified by title alone will merge two genuinely different works that share a title.
• [5] the write. This is the step that cannot be undone, and notice that it is a whole-file overwrite: 412 lines in, 380 out. Nothing in the trace shows which 32 lines went.
• [7] the report. Two claims. "Removed 6 duplicates" and "no other changes". The second is the one nobody reads and the one most likely to be wrong, because a whole-file rewrite can change formatting, ordering and encoding without anybody intending it.

The right intervention is not more scrutiny at step 7. It is a copy of the file before step 5, which converts the bottom row of the table into the top row, and is the single most useful habit in this unit.

Working with agents, practically

Move actions up the table before you start. Work on a copy. Use a branch. Send to yourself first. Almost every irreversible action has a reversible rehearsal, and arranging one is cheaper than checking carefully.

Bound the run. A loop with a stopping condition you set is a different object from one that runs until it decides it is finished. Say how many steps, which tools, and what it must not touch.

Ask for the plan before the actions. Approving a plan is not verification, and it will not catch an error at step 4. But it catches the wrong-file class of error at the point where it is still free.

Give it the least it needs. An agent can only do what its credentials and its tool list allow. A key that can read but not write, a tool list without the destructive operation, a path or domain it is confined to — each of these turns a class of bottom-row action into one that cannot happen at all. This is worth more than any amount of checking, because it removes the possibility rather than detecting the result.

Ask for a dry run. Many operations have a mode that reports what they would do without doing it. Where one exists, the run becomes free to undo.

Check the calls, not the story. Every time.

None of which is an argument against agents. They do real work, and much of it sits in the top two rows of the table, where letting the loop run unattended is exactly right. Nor is a trace worthless — it is the most inspectable thing any of these systems gives you, which is why it is worth learning to read. The claim is narrower: the record part is evidence, the narration part is not, and the two arrive looking identical.

Summary

• An agent acts rather than answers, and what changes is that the world may change before you have looked at anything. Checking afterwards describes an error; it does not prevent one.
• No step is the safe one. The plan, the choice of arguments, the decision to continue and the final report are generated; the execution depends on state, timing and permissions, and can fail or half-succeed on its own account.
• A result is a claim about a call. Whether the world is as intended is a separate question, answered by reading the state back.
• Sort actions by what recovery costs, not by how reliable the system seems. Reliability changes how often you need the remedy, not what it costs.
• In a trace, calls and results are records if the trace is the system's own log. A rendering in a chat window may summarise, reorder or omit steps, and a narrated account of a run is generated text. Know which one you are reading.
• Move an action up the reversibility table before you start — a copy, a branch, a dry run — rather than checking an irreversible one harder.

Then answer cp11-reversibility and cp11-trace-evidence at the foot of this page.