Unit 4: Ask AI to generate the routine

Reading time: about 9 minutes. Activity: about 12 minutes, plus optional extensions. Course items: see course/course-map.md for items 2 and 8.

Prerequisite: Unit 3. This unit assumes you have derived B4 yourself and can state which convention this module uses. If you have not, go back — the whole point of the ordering is that your reference exists before the generated output does.

What you should be able to do afterwards

1. Recognise what an underspecified prompt leaves for the generation to decide.
2. Name the failure modes this module expects in generated Bernoulli code, using the module's own codes.
3. Separate the parts of a generated response that a test could settle from the parts only a source could settle.
4. Produce a saved artefact — prompt, output, date, and your own triage — that Unit 5 will check and Unit 8 will use as evidence.

The prompt

Here is the prompt for this unit, in full:

Write a Python function that returns Bernoulli numbers.

That is the entire thing. One sentence, no context, no requirements.

It is important that you do not read this as a trick. It is not a deliberately bad prompt constructed to make a point; it is the sentence people actually type. When you know roughly what you want and the tool is open in front of you, this is what gets written, and a good deal of the time it produces something that works well enough. That is precisely why it is worth examining.

The precise prompt is not in this unit

There is a precise version of this prompt. It states the interface, the convention, the return type, the range and the behaviour at odd indices. You will build it in Unit 6.

It is deliberately withheld until then, and this page is telling you so rather than quietly omitting it. The reason is straightforward: if you see a good specification before you have felt the need for one, you learn that good specifications exist, which you already knew. If you write the vague prompt first, look at what comes back, and only then see the precise version, you learn what each clause of it is for — because you watched the failure it prevents.

What the sentence leaves open

Read the prompt again and count the decisions it does not make.

• Which convention? B1 = -1/2 or B1 = +1/2. Both standard, as Unit 3 established.
• What return type? Exact rationals, or floating-point decimals.
• What range of indices? Starting where, running how far.
• What happens at the odd indices? Exact zeros, or absent, or something else.
• Does exactness matter? Nothing in the prompt says it does.

Somebody has to settle every one of those before the code can exist. You settled none of them.

This is the mechanism worth holding on to. A language model produces a statistically plausible continuation of the text it is given. Where your text is silent, the continuation still has to contain something — a function body cannot be partially absent. What it contains reflects the patterns in similar code the model was trained on, not your requirements, because your requirements were never stated.

So an unstated requirement does not appear in the output as a gap or a question. It appears as a confident default. That is much harder to notice than a gap would be, and it is the reason this unit exists.

The failure modes to expect

The module names five failure modes in generated Bernoulli code. These are the codes the checker emits in Unit 5, so the names are worth learning as they stand.

A few notes on each.

b1-sign follows directly from the missing convention clause. The resulting routine is not broken — it is correct under a different, equally published convention. It disagrees with this module at exactly one index and nowhere else.

odd-terms matters more than it looks. B3 is not approximately zero; it is zero. A routine that omits the odd indices, or returns a tiny non-zero decimal for them, has failed a property that holds exactly.

ada-indexing becomes noticeably more likely if your prompt mentions Ada Lovelace or Note G, because text about Note G uses her numbering. A routine produced this way is a faithful reading of the historical source and a broken modern component at the same time.

off-by-one is the one that most resists being read. Every value returned is a genuine Bernoulli number, computed correctly and represented exactly. Each is filed under the wrong index. There is nothing in the code for a careful reader to catch, because the error only exists relative to a specification.

not-fraction is the default outcome of asking for numerical Python, because most numerical Python is float-based. For small indices it is convincing: -0.0333… really is close to -1/30. Then the denominators grow. B30 is 8615841276005/14322, and accumulated rounding error is no longer invisible. A test that checked B0 to B12 with a tolerance would pass such a routine.

What these five have in common is the important part: none of them makes the code look suspicious. They are invisible to reading and visible to testing. That asymmetry is the entire argument for the verification lab in Unit 5.

The checker recognises three further conditions — raises, import-error and no-function — which describe code that does not run at all. Those announce themselves, so this unit does not dwell on them.

The sixth failure mode, which no test catches

Generated code frequently arrives with explanation attached. Comments, a docstring, a paragraph of context. Something along these lines:

# Ada Lovelace used this same recurrence in Note G (1843).
def bernoulli(n):
    ...

Suppose you check the arithmetic and it is correct. What does that tell you about the comment?

Nothing at all.

The code and the comment were produced by the same process, and they fail independently. The code got checked because code is checkable — you can run it against known values. The historical claim sat directly beside it, phrased with the same confidence, dated with the same specificity, and was checked by nobody.

This is what makes generated output awkward to evaluate: it is a mixture of things with completely different failure modes. Arithmetic you can test. Assertions about the world you can only look up. Both parts can be equally polished, so surface quality alone does not tell you which claim is reliable.

The module is not claiming that the example comment above is false. The point is sharper than that: you do not know, and the correctness of the code adjacent to it is not evidence either way. Unit 7 deals with what is actually established about Note G, and course/historical-notes.md is careful about which parts are settled and which are contested. Bring that same care to any historical claim that arrives inside generated code.

This is checkpoint cp04-trust-triage.

A worked example: a claim that does not survive checking

The same triage applies to claims that arrive with no code attached at all. Here is one phrased with total confidence:

Gödel's theorem proves that an AI cannot verify its own output.

You do not need formal logic to triage this sentence. It names a theorem about one kind of system, then applies it to the broad category "an AI" without showing that the theorem's conditions or conclusion transfer. Mark the theorem statement for a source check and the transfer for a separate argument. Unit 9 defines the relevant formal terms, states Gödel's and Tarski's results within their actual scope, and explains why neither directly settles what a language model can verify.

What to do with the output

Save it. Exactly as generated, without tidying it up. Save the prompt too, word for word, and the date you ran it. The untidied version is the evidence; a cleaned-up version is a summary of the evidence, which is not the same thing.

You will need it twice more:

• Unit 5 runs it through the checker and names what is wrong with it.
• Unit 8 uses it in your evidence pack, alongside your verification result and your correctness defence.

Do not run it yet. Two reasons. The first is pedagogical: your predictions are far more informative if they are written down before the checker gives you the answer. The second is practical, and it holds outside this module too.

Safety. Generated code is code from a source you cannot vouch for. Read it before you run it, and run it in a disposable local environment. The checker in Unit 5 imports and executes the file you give it, which is a real consideration and not a formality.

If you do not have access to an AI system

Use the sample implementations in course/lab/examples/:

• wrong_b1_sign.py
• wrong_odd_terms.py
• float_output.py
• off_by_one_range.py
• correct_example.py

These are real implementations exhibiting the failure modes above, and the activity works identically with them. This is an equal route, not a substitute: in one respect the supplied examples are the better teaching set, because every failure mode is guaranteed to be present and one of the five files is correct, so the exercise of telling them apart is genuine.

The same applies if you have access but would rather not use it, or if what you generate turns out to be flawless. A correct generated routine is a perfectly good result; take one of the examples as a second specimen and triage that too.

Privacy note

Do not paste confidential, personal, unpublished or assessment-restricted material into an AI system. Nothing in this unit requires it — the prompt is one sentence about a published mathematical sequence. The wider course covers this properly; see course/course-map.md for item 5.

Check yourself

This unit has one checkpoint:

• cp04-trust-triage — what you can trust before testing

python3 selfcheck.py run cp04-trust-triage    # or use the browser self-check

Both routes ask the same question and give the same feedback. It is about the comment, not the code, which tends to be the surprising part.