Unit 10: Re-prompting, specification, and overreliance

Reading time: about 16 minutes. Activity: about 29 minutes.

1. The claim

There is a widespread idea that better results come from better wording, and that somewhere there exists a phrase that unlocks a correct answer. This unit argues something less exciting and considerably more useful.

Better prompting is precise requirements work. It is the same skill as writing a specification. If your route took you through Units 7 and 8, you have already been doing it: every failure mode the checker named in Unit 8 was telling you which requirement you had failed to write down. If your route skipped them, this page shows the same thing in one place.

2. Where you started

On the routes that include them, Unit 7 has you ask a system for a routine that returns Bernoulli numbers (the sequence Unit 6 introduces), and Unit 8 checks what comes back against a reference. The prompt Unit 7 began with was:

Write a Python function that returns Bernoulli numbers.

Every word of that is true, and none of it settles what a correct answer would be. It leaves at least five decisions open:

• Which convention? B1 = -1/2 or B1 = +1/2. Both are standard and published.
• Which indexing? Modern indexing, where every index is numbered including the zero ones, or Lovelace's, where only the non-zero terms carry labels.
• Which return type? An exact Fraction, a float, or a formatted string.
• Which domain? Is bernoulli(0) defined? bernoulli(1)?
• The zero terms: returned as Fraction(0), or skipped, or an error.

Something has to settle those five decisions. If the prompt does not, the output does, according to patterns in what the system was trained and tuned on, not your needs. The result is fluent and plausible; whether it is correct is a question the prompt never asked.

3. The failure modes were the missing requirements

Put the Unit 8 diagnoses beside that prompt -- the codes its checker returns, one for each way a generated routine went wrong -- and each points at a requirement:

Not one of those is a wording problem. Each points at a requirement the prompt never stated, and stating it is what makes the failure a failure, whether it came from a default or from an ordinary bug. That table is the substance of this unit, and cp10-spec-clauses has you match three of its clauses to the failures they prevent.

4. The precise prompt

If your route skipped Units 6 to 8, these terms are enough to read it. The Bernoulli numbers are a fixed sequence of fractions B0, B1, B2, ...; the number after B is the index, the position in the sequence. Two published conventions differ only at B1, every odd index above 1 has the value zero, and Lovelace's Note G labelled only the non-zero values. A Fraction is an exact fraction such as 1/6; a float is a decimal approximation such as 0.1666667.

Here it is in full. It is not a cleverer sentence; it is the same request with the decisions written down. The clause labels are for the activity and are not part of what you would send.

(C1) Interface. Write a single Python module that defines exactly this function at the top level of the module:

text bernoulli(n: int) -> fractions.Fraction

(C2) Convention. Use the first Bernoulli numbers, in which B1 = -1/2. Do not use the second convention, in which B1 = +1/2.

(C3) Indexing. Use modern indexing, in which every index is numbered, including the odd indices whose value is zero. Do not use the nineteenth-century numbering of Ada Lovelace's Note G, in which only the non-zero terms are labelled.

(C4) Domain. bernoulli(n) must be defined for every integer n >= 0. For n < 0, raise ValueError.

(C5) Zero terms. For odd n > 1, return Fraction(0). Do not skip those indices, omit them, or raise.

(C6) Arithmetic. Return exact fractions.Fraction values only. Never return float, Decimal, or a string, and do not use floating-point arithmetic anywhere in the computation.

(C7) Anchor values. These values must be exactly reproduced: B0 = 1, B1 = -1/2, B2 = 1/6, B3 = 0, B4 = -1/30, B12 = -691/2730, B30 = 8615841276005/14322.

(C8) Importability. The module must import with no side effects: no printing, no file or network access, and no third-party packages — Python standard library only.

(C9) Comments. Do not include historical claims about Ada Lovelace or Note G in comments unless I ask for them. I cannot test a comment.

The interface line in C1 is quoted exactly as the checker's contract states it. That is the point of the whole unit in one line: a specification precise enough to test against is a specification precise enough to prompt with. If you find you cannot write the prompt, it is usually because you did not yet know what you wanted.

Two clauses deserve a note.

C7 is a trap as well as a requirement. Naming the acceptance values makes the requirement concrete, and it also makes it possible for generated code to satisfy them with a lookup table that fails at every other index. Examples are not a specification. That is why the checker compares thirty-one indices rather than the seven you listed, and why C2 to C6 state properties rather than only values.

The lookup table is one case of a general failure: a requirement that something useless also meets. It happens far from Bernoulli numbers. A protocol is a fixed exchange of messages, for example to agree a secret key between two parties. One 2026 preprint proves, with a program checking every step, that a protocol keeps its key secret. Its authors note that their definition of "secure" alone would also accept a protocol that always aborts: it stops without producing a key. So, following earlier work, they add a limit on how often an honest run, one with no attacker, may abort [L18, abstract, sec. III.B]. When you write a clause, ask what the laziest output that passes it would be, and add what rules it out.

Where you can, make the wrong thing impossible to write. A clause asks; a representation can enforce. The same preprint's authors found that an AI-generated protocol worked out the key from both parties' data combined, which no single party held. In response they built a language in which each party can act only on what it holds [L18, sec. II]. That makes the error impossible to write, instead of asking more firmly. A type, a schema or a restricted set of operations can rule out a whole class of error before any output exists. It rules out only that class: say what the construction does not enforce, and check that separately. And the representation is then part of the specification, and needs validating like the rest of it.

C6 asks for more than the checker can see. The checker observes what comes back: the type of every returned value and whether it equals the oracle exactly. "Do not use floating-point arithmetic anywhere in the computation" is about the inside of the routine, and a program can compute in floats and convert before returning. What the checker can establish is the observable half — exact Fraction values that agree over the tested range. The rest is a source-review requirement, which is a different kind of check with different limits: reading establishes what the code appears to do, not what it does.

That gap is worth noticing rather than papering over. Several clauses have it: C4's ValueError for a negative n lies outside the indices the checker tries (0 to 30), and C8's "no side effects" does not show in the values returned. A dynamic check on outputs cannot establish which algorithm was used, how the routine scales, that no lookup table is hidden inside it, or that it behaves outside the range that was tested.

No better checker would close the gap in general. Here decide has its technical sense: to give the correct yes-or-no answer in every case, in a finite time. A question no program can decide is undecidable. Rice's theorem (1953) [T11, pp. 364-365], [T13, sec. 3.2] is about properties of what a program does that some programs have and others lack. It says that no checking program can always decide, for every program given to it, whether that program has such a property over all inputs. Roughly: no general tool answers every interesting question about what programs do. The best-known example is the halting problem: whether a program eventually stops, or runs for ever, on a given input. It is a close variant of the problem Turing proved undecidable in 1936-7 [T10, sec. 8], and it is undecidable too [T12, sec. 2.4.3]. So no checker can decide such properties for every program it might be given. This lab tests a finite range; a particular program can sometimes be proved correct for all inputs, but that proof is about that program, which is why a certificate speaks for one artefact rather than for a method. If either result is new to you, the glossary entry decidable sets out both, and Unit 18, on the complete route, returns to them.

C9 is not enforced by any check. No code in the lab reads comments. It is in the prompt because a historical claim in a code comment is generated the same way as the code, fails independently of it, and passes every test in the course. Unit 16 is about exactly that.

Meeting the contract is not the same as having the right contract. The checker verifies: it establishes that a routine meets the contract over the tested range, B1 = -1/2 included. Whether B1 = -1/2 was the right thing to specify is validation, and no check against the contract can answer it, because the contract is what the check compares with. You validate by using the numbers for what they are for. Unit 6's sums-of-powers example (on the routes with the mathematics) does exactly that: the same formula gives a different sum under each convention, and the sum you need is what settles which convention to write into C2.

5. Most of these clauses are bugs that already happened

Read the precise prompt once more and notice what most of it is: a list of things that can go wrong. The failures below are the lab's examples, written for the lab to show each one. They are not observed model outputs, and this course has no measurement of how often a system produces any of them.

• C2 exists because a routine can come back correct under the second convention.
• C3 exists because a routine can come back numbering only the non-zero terms, faithfully reproducing Note G.
• C5 exists because a routine can come back with something other than zero at an odd index: the shifted example returns -1/30 at n = 3.
• C6 exists because a routine can come back in floating point, where B3 is -1.11e-16 rather than 0.
• C7 exists because a routine can come back shifted by one index: naming the values at fixed indices pins the sequence in place.
• C4 was written before its bug. None of the supplied examples raises at n = 0, but a routine that did would fail the specification, and the checker's raises code exists for it.

Of these six clauses, five answer a failure the lab's examples show. One, C4, answers a failure that was foreseen. Many lasting requirements are lessons written down after earlier failures in the same way. That is what separates specification from prompt engineering as it is usually described: you are not searching for magic words, you are writing down a requirement each time reality teaches you one, or before it has to. Requirements also come from elsewhere: what a user needs, what the law or a standard demands, what a hazard analysis turns up. Those clauses are just as real, even before anything has gone wrong. A specification accumulates in much the way a test suite does — often one clause per fault that reached you — and it can be reviewed, argued with, and handed to someone else. What a precise prompt brings back is still a candidate, to be checked independently like the first one.

6. Overreliance

Overreliance is not using AI. It is relying on generated output past the point where you can check it. Generated output is the case this course is about, not the only one: the same goes for any source or tool you rely on past that point -- a calculator, a search result, a colleague's answer, a reference book. The distance between what you rely on and what you can check is your own understanding, which makes overreliance situational: the same use is sound for a person who can evaluate the result and unsound for a person who cannot.

On the routes with the mathematics, Unit 6 has you work out B4 by hand. Five binomial coefficients, a few fractions, no calculator, and out comes -1/30. It was slow, and it looked like a detour on the way to the interesting part.

It was the load-bearing step. It is the reason that if a routine returns 1/30, you know something is wrong rather than assuming you have misremembered. Without that derivation, or some other reference independent of the output, -1/30 and 1/30 are two equally plausible strings, and you have no basis to prefer either. The oracle, the checker and the property test only convert into a judgement if there is someone who can read the result and act on it.

This is why the course put the mathematics before the generation, and why cp06-b4-by-hand never reveals its answer on a wrong attempt. A value you derived independently is strong evidence because the generated answer did not determine it. Authoritative sources, measurements, and separately implemented methods can play the same role in other tasks.

A correct answer is not evidence that you learned

Overreliance has a second form, and it matters most while you are still learning. Checking an answer tells you whether the answer is right. It does not tell you whether you can now do the task, and the two come apart. In one field experiment, school students who practised mathematics with a standard chat assistant did far better on the practice problems, and then, once the assistant was taken away, scored 17% lower on the exam than students who had never had it; a version set up to give hints rather than answers largely avoided the drop [E21, "Experimental Design", "Main Results"]. That was one setting, with school students, and research on how these tools affect learning is still early. What the experiment shows is why assisted performance and unaided performance need separate checks. This course's response is its own order: attempt first, then use the system to check or challenge your attempt, not in place of it. In a survey published in 2026, of UK full-time undergraduates polled in December 2025, 94% said they use generative AI to help with assessed work [E22, pp. 4, 9]; for them, at least, the question is how, not whether.

The practical test is short. After you have checked an assisted answer, put the system away and do a comparable case on your own. If you cannot, the answer was right and the learning has not happened yet.

When to stop

There is a practical rule in this. Pause and re-ground when you can no longer understand and defend what is being produced. The signal is concrete, not a mood. Pause if you cannot:

1. predict relevant behaviour;
2. explain the transformation from input to output at the level your claim needs;
3. name a suitable independent check; or
4. interpret a failed check well enough to decide what to revise or escalate.

Subjective confidence is not evidence that any of these capacities remains. Narrow the request until you can follow it, rebuild the missing foundation (as you rebuilt B4 by hand), or seek qualified review. Do not keep expanding a result that you can no longer evaluate.

7. The honest position

This is not an argument against using these systems.

What you gain is real. The precise prompt plus the oracle is genuinely faster than writing the routine from scratch, and the specification you wrote is useful work in its own right — you would need it to brief a colleague, and you would need it to test anything at all.

Two things do not transfer. Responsibility for the result stays with you, whatever produced it. So does judgement: someone has to know what correct looks like, and in your work that someone is you.

And note the luxury of this example. Bernoulli numbers have an oracle. Most of the questions you will put to an AI system do not — there is no reference implementation for the summary of a document, the argument in an essay, or the interpretation of a policy. This case was chosen because it is clean, not because it is typical.

When there is no oracle, you still have the same moves in weaker form:

• Write the specification anyway. It is the part that makes anything checkable.
• Ask for checkable evidence rather than an answer: a locator and the quoted passage, a worked step you can redo by hand, test output against values you derived yourself. Checking such a certificate is usually far cheaper than producing the result: checking B4 by the recurrence took minutes; getting a correct routine took several units.
• Check claims against independent sources rather than against the fluency of the output.
• Produce a second, independent version and compare — by another method, another tool, or another person. Asking the same system "are you sure?" is not one: in a study published in 2024, assistants from 2023 that were trained partly on people's ratings shifted towards what the user appeared to believe. That question alone could turn a correct answer into an incorrect one [L5, secs. 3.2-3.3]. It tests the system's response to pressure, not the answer.
• Check a relation where you cannot check a value: reword the question or reorder the sources, and claims the sources settle should not change. Software testers call such a relation metamorphic [E20, sec. 2.2]; Unit 5 has the details.
• State plainly what you were unable to verify, rather than letting confident prose stand in for a check.

The activity applies those moves to a citation, a supplied policy passage, and a recommendation containing an unstated value judgement. In each case you must name the claim, an independent check, the judgement that remains, and the point at which you would stop or seek qualified human review.

A note on the general case

There is a general version of what C1 to C9 are doing. "Correct" is not a property a value carries on its own: B1 = -1/2 and B1 = +1/2 are both correct, and they appear to disagree only because the convention each assumes has been left out. A specification clause supplies exactly that missing information: the convention, the range, the type. That is what makes two answers comparable at all, before you ask which one to use. That argument in general form is in Second-Order Reasoning: The Lens Behind The Course, which the closing Unit 18 teaches from; neither this page, the activity, nor cp10-spec-clauses depends on it.

Checkpoint

Finish this unit at cp10-spec-clauses, by either route:

python3 selfcheck.py run cp10-spec-clauses  # or in the browser