Unit 6: Re-prompting, specification, and overreliance

Reading time: about 10 minutes. Activity: about 20 minutes.

Course items: 2 (how to evaluate AI outputs) and 4 (AI and critical thinking). See ../course-map.md.

The claim

There is a widespread idea that better results come from better wording, and that somewhere there exists a phrase that unlocks a correct answer. This unit argues something less exciting and considerably more useful.

Better prompting is precise requirements work. It is the same skill as writing a specification. You have already been doing it: every failure mode the checker named in Unit 5 was telling you which requirement you had failed to write down.

Where you started

The prompt Unit 4 began with was of this kind:

Write a Python function that computes Bernoulli numbers, like the ones in Ada Lovelace's program.

Every word of that is true, and none of it is decidable. It leaves at least five decisions open:

• Which convention? B1 = -1/2 or B1 = +1/2. Both are standard and published.
• Which indexing? Modern indexing, where every index is numbered including the zero ones, or Lovelace's, where only the non-zero terms carry labels.
• Which return type? An exact Fraction, a float, or a formatted string.
• Which domain? Is bernoulli(0) defined? bernoulli(1)?
• The zero terms: returned as Fraction(0), or skipped, or an error.

Something has to settle those five decisions. If the prompt does not, the output does, according to whatever is most frequent in the text the system was trained on. The result is fluent and plausible; whether it is correct is a question the prompt never asked.

The failure modes were the missing requirements

Put the Unit 5 diagnoses beside that prompt and they stop looking like coding mistakes:

Not one of those is a wording problem. Every one is a missing requirement. That table is the substance of this unit, and matching its two columns is checkpoint cp06-spec-clauses.

The precise prompt

Here it is in full. It is not a cleverer sentence; it is the same request with the decisions written down. The clause labels are for the activity and are not part of what you would send.

(C1) Interface. Write a single Python module that defines exactly this function at the top level of the module:

text bernoulli(n: int) -> fractions.Fraction

(C2) Convention. Use the first Bernoulli numbers, in which B1 = -1/2. Do not use the second convention, in which B1 = +1/2.

(C3) Indexing. Use modern indexing, in which every index is numbered, including the odd indices whose value is zero. Do not use the nineteenth-century numbering of Ada Lovelace's Note G, in which only the non-zero terms are labelled.

(C4) Domain. bernoulli(n) must be defined for every integer n >= 0. For n < 0, raise ValueError.

(C5) Zero terms. For odd n > 1, return Fraction(0). Do not skip those indices, omit them, or raise.

(C6) Arithmetic. Return exact fractions.Fraction values only. Never return float, Decimal, or a string, and do not use floating-point arithmetic anywhere in the computation.

(C7) Anchor values. These values must be exactly reproduced: B0 = 1, B1 = -1/2, B2 = 1/6, B3 = 0, B4 = -1/30, B12 = -691/2730, B30 = 8615841276005/14322.

(C8) Importability. The module must import with no side effects: no printing, no file or network access, and no third-party packages — Python standard library only.

(C9) Comments. Do not include historical claims about Ada Lovelace or Note G in comments unless I ask for them. I cannot test a comment.

The interface line in C1 is quoted exactly as the checker's contract states it. That is the point of the whole unit in one line: a specification precise enough to test against is a specification precise enough to prompt with. If you find you cannot write the prompt, it is usually because you did not yet know what you wanted.

Two clauses deserve a note.

C7 is a trap as well as a requirement. Naming the acceptance values makes the requirement concrete, and it also makes it possible for generated code to satisfy them with a lookup table that fails at every other index. Examples are not a specification. That is why the checker compares thirty-one indices rather than the seven you listed, and why C2 to C6 state properties rather than only values.

C9 is not enforced by any check. No code in the lab reads comments. It is in the prompt because a historical claim in a code comment is generated the same way as the code, fails independently of it, and passes every test in the module. Unit 7 is about exactly that.

Every clause is a bug that already happened

Read the precise prompt once more and notice what it actually is: a list of things that went wrong.

• C2 exists because a routine came back correct under the second convention.
• C3 exists because a routine came back numbering only the non-zero terms, faithfully reproducing Note G.
• C5 exists because a routine skipped the odd indices instead of returning zero.
• C6 exists because a routine came back in floating point, where B3 is -1.11e-16 rather than 0.
• C4 exists because a routine raised an exception at n = 0.

Every clause in a good specification is a bug that already happened. That is what separates specification from prompt engineering as it is usually described: you are not searching for magic words, you are writing down a requirement each time reality teaches you one. A specification accumulates in exactly the way a test suite does — one clause per fault that reached you — and it can be reviewed, argued with, and handed to someone else.

Overreliance

Overreliance is not using AI. It is relying on generated output past the point where you can check it. The distance between the two is your own understanding, which makes overreliance situational: the same use is sound for a person who can evaluate the result and unsound for a person who cannot.

Think back to working out B4 by hand in Unit 3. Five binomial coefficients, a few fractions, no calculator, and out comes -1/30. It was slow, and it looked like a detour on the way to the interesting part.

It was the load-bearing step. It is the reason that when a routine returned 1/30, you knew something was wrong rather than assuming you had misremembered. Without that half-hour, -1/30 and 1/30 are two equally plausible strings, and you have no basis to prefer either. The oracle, the checker and the property test only convert into a judgement if there is someone who can read the result and act on it.

This is why the module put the mathematics before the generation, and why cp03-b4-by-hand never reveals its answer on a wrong attempt. A value you derived independently is strong evidence because the generated answer did not determine it. Authoritative sources, measurements, and separately implemented methods can play the same role in other tasks.

When to stop

There is a practical rule in this. Pause and re-ground when you can no longer understand and defend what is being produced. The signal is concrete, not a mood. Pause if you cannot:

1. predict relevant behaviour;
2. explain the transformation from input to output at the level your claim needs;
3. identify a suitable independent check; or
4. interpret a failed check well enough to decide what to revise or escalate.

Subjective confidence is not evidence that any of these capacities remains. Narrow the request until you can follow it, rebuild the missing foundation (as you rebuilt B4 by hand), or seek qualified review. Do not keep expanding a result that you can no longer evaluate.

The honest position

This is not an argument against using these systems.

What you gain is real. The precise prompt plus the oracle is genuinely faster than writing the routine from scratch, and the specification you wrote is useful work in its own right — you would need it to brief a colleague, and you would need it to test anything at all.

Two things do not transfer. Responsibility for the result stays with you, whatever produced it. So does judgement: someone has to know what correct looks like, and in your work that someone is you.

And note the luxury of this example. Bernoulli numbers have an oracle. Most of the questions you will put to an AI system do not — there is no reference implementation for the summary of a document, the argument in an essay, or the interpretation of a policy. This case was chosen because it is clean, not because it is typical.

When there is no oracle, you still have the same moves in weaker form:

• Write the specification anyway. It is the part that makes anything checkable.
• Check claims against independent sources rather than against the fluency of the output.
• Produce a second, independent version and compare — by another method, another tool, or another person.
• State plainly what you were unable to verify, rather than letting confident prose stand in for a check.

The activity applies those moves to a citation, a supplied policy passage, and a recommendation containing an unstated value judgement. In each case you must name the claim, an independent check, the judgement that remains, and the point at which you would stop or seek qualified human review.

A note on the general case

There is a general version of what C1 to C9 are doing. "Correct" is not a property a value carries on its own: B1 = -1/2 and B1 = +1/2 are both correct, and they appear to disagree only because the convention each assumes has been left out. A specification clause supplies exactly that missing index — the convention, the range, the type — and it is what makes two answers comparable at all, before any question of which one wins. That argument in general form is in course/second-order.md, which the closing Unit 9 teaches from; neither this page, the activity, nor cp06-spec-clauses depends on it.

Checkpoint

Finish this unit at cp06-spec-clauses, by either route:

python3 selfcheck.py run cp06-spec-clauses    # or use the browser self-check

Full instructions are in activity.md.