Unit 8 activity: Verification lab

Release candidate. The unit videos are added in the first release.

Time: about 27 minutes. Work through Route A or Route B. They are equal routes and either one completes the unit; do both if you have time, because they catch different things.

Keep everything you write down. On the routes that include Unit 17 it goes into your evidence pack; on the short route this record is where the course ends, so finish it here.

Before you start: the safety step

This step belongs to both routes and comes first.

check.py executes the file you give it, in a separate process with a timeout and resource limits — which limits the damage an accident can do, and does not make unread code safe. Your Unit 7 code came from a source you did not write and have not read. Read it now, before you do anything else, and answer these three questions in writing:

  1. Which modules does it import? A Bernoulli routine needs fractions and possibly math. Anything else needs a reason.
  2. Does anything happen at import time — outside the function body — such as printing, writing a file, opening a network connection, or running a subprocess?
  3. Is there anything you cannot explain? Encoded strings, eval, exec, os.system, or an install command.

If any answer worries you, do not run it. Delete the suspicious part, or switch to Route B, which never executes anything.

If you did not read it, or you have no reason to trust where it came from, run the lab in a container or a virtual machine rather than on your own account — or take Route B, which executes nothing. If you have no container or virtual machine, take Route B. Not as an administrator, either way.

Route A — Automated verification

A1. Set up (2 minutes)

cd course/lab
python3 reference_bernoulli.py

Confirm the final line reads:

OK: oracle agrees with all 22 published values, odd terms are zero.

If it does not, stop. The oracle is not verified, so nothing you check against it means anything.

A2. Meet the diagnoses (4 minutes)

Run the checker against each supplied wrong example, and against the correct one:

python3 check.py examples/wrong_b1_sign.py
python3 check.py examples/ada_indexing.py
python3 check.py examples/float_output.py
python3 check.py examples/off_by_one_range.py
python3 check.py examples/correct_example.py

Fill in this table from the output before you check the marking guidance below:

file first divergence at n expected got diagnosis code(s)
wrong_b1_sign.py
ada_indexing.py
float_output.py
off_by_one_range.py
correct_example.py — — —

Then answer in one sentence each:

Then run the fifth wrong example, python3 check.py examples/wrong_b4_sign.py, and note what its report does not contain.

A3. Check your own code (5 minutes)

Put the routine you saved in Unit 7 into the exercise stub, or check it where it is. Either works:

python3 check.py exercises/ex04_bernoulli.py

or, for a file anywhere on your machine:

python3 check.py /full/path/to/your_file.py

Record:

Against your Unit 7 prediction. Which codes and which first index did you predict? Write one sentence on what you got right and what your reading missed.

If your file passed on the first run, do not skip ahead. Run it again with the range stated and confirm the report says it compared every n from 0 to 30:

python3 check.py exercises/ex04_bernoulli.py --max-n 30

Thirty is the default, and it is not arbitrary: B30 is the largest index in the published values the oracle is checked against, so a comparison beyond it is against an oracle no published table has confirmed there.

Then write one sentence saying which clause of the specification your Unit 7 prompt happened to get right by luck rather than by stating it.

A4. Repair (7 minutes)

You have two ways to fix this, and the choice is the interesting part.

Do at least one. Note which requirement was missing — that observation is exactly what Unit 10 builds on. Then re-run the checker with --verbose until you see:

PASS - agrees with the oracle for every n from 0 to 30,
       returns exact Fractions, and odd terms above B1 are zero.

Passing is not the end of the job. You still have to be able to say
*why* it is correct. That is the Unit 17 defence.

(Indices compared: 0 to 30)

A5. One last question

The checker passed. Write down, in one sentence, something that is still not established about your routine by that pass.

Route B — Table-based verification

This route uses no Python. It performs the same comparison by hand, and it completes the unit on its own terms.

B1. Get the reference values (2 minutes)

Use the reference table in the unit page. If you have Python available and want the machine-printed version, python3 reference_bernoulli.py prints the same data, but this is optional here.

B2. Get the generated values (4 minutes)

If you saved a list of values in Unit 7, start from it and carry it into B3 as your generated column. If you saved a program, or your list stops short of B12 or gives no B30, go back to the AI system you used and ask for the values rather than a program:

List the Bernoulli numbers B0 to B12 as exact fractions, and also give B30.

Copy what comes back exactly as it is given, including any decimals. Do not tidy it up. Tidying is where evidence gets lost.

If you have no AI access, use a supplied specimen instead, just as Unit 7 does:

  1. Open course/lab/examples/wrong_b1_sign.py in any text editor. You are reading it, not running it — this route stays code-free.
  2. Its docstring states exactly what the routine returns: it is "correct under the second Bernoulli convention (B1 = +1/2)". So its output table is the reference table with precisely one entry changed: 1/2 at n = 1.
  3. Write that out as your "generated value" column — every reference value unchanged except 1/2 at n = 1 — and carry it into B3.

B3. Compare in index order (6 minutes)

Write the two columns side by side, in index order, and go down the rows.

n reference value generated value same?
0 1
1 -1/2
2 1/6
3 0
4 -1/30
5 0
6 1/42
7 0
8 -1/30
9 0
10 5/66
11 0
12 -691/2730
30 8615841276005/14322

Stop at the first row that differs. That is the first divergence, found by hand, using the same method the tool uses. Record it:

Against your Unit 7 prediction. Which codes and which first index did you predict? Write one sentence on what you got right and what your reading missed.

Then check three things that are easy to miss when scanning:

  1. Are the odd rows present and exactly 0, or are they missing, or "about zero"?
  2. Are the values exact fractions, or decimals such as -0.0333?
  3. Did B30 come back at all, and is it exactly 8615841276005/14322?

The page's checks that need no reference table test the pasted values on their own: do the even-indexed signs alternate, are the denominators right (try B12), is B30 the right size, and do the first few rows sum to zero when substituted into Unit 6's identity?

B4. Repair (6 minutes)

Go back to the AI system, state the requirement that was missing, and ask again. Compare the new table against the reference the same way. Record which single sentence you added and what changed.

If you have no AI access, treat a second supplied specimen as the "regenerated" attempt:

  1. Open course/lab/examples/ada_indexing.py and read its docstring — again without running anything.
  2. Build its output column from what the docstring states: the routine uses Lovelace's Note G labels, so her B1, B3, B5, B7 are the modern B2, B4, B6, B8, and her even indices carry nothing. Its column therefore reads 1 at n = 0, then 1/6, 0, -1/30, 0, 1/42 at n = 1 to 5.
  3. Compare that column against the reference exactly as in B3, in index order, and record the first divergence.
  4. Instead of the sentence you added, record the single sentence a repair prompt would need for this specimen — the requirement its docstring says it violates.

B5. One last question

Suppose a candidate's table agreed with the reference at every index. Write down, in one sentence, why that would not establish that a program produced those values.

B6. Check what the table evidence supports (4 minutes)

Open cp08-table-evidence in the browser self-check. It gives four observations from comparing a candidate's table with the reference by hand. Match each observation to the strongest conclusion it supports.

This is the browser-assessable counterpart to the executable checkpoint, not a simulation of running code. It checks modern indexing, zero odd terms, exact displayed values, first-convention handling, and the boundary of manual evidence. A matching table supports the cells compared; it does not establish the return type of unseen code, unlisted indices, or that a program produced the table.

Finish the unit at the checkpoints

The checkpoints are formative and unmarked. cp08-diagnose and cp08-what-it-shows are shared: the first names the failure, the second asks what your check did and did not establish, which is the same question on either route. cp08-table-evidence records the manual evidence objective on Route B. cp08-bernoulli runs check.py against a file and belongs to Route A. The two route-specific checkpoints test the same contract with honestly different coverage.

Python route — from course/lab:

python3 selfcheck.py run cp08-bernoulli
python3 selfcheck.py run cp08-diagnose
python3 selfcheck.py run cp08-what-it-shows

To practise both routes, including the table-evidence checkpoint, run the whole unit:

python3 selfcheck.py run --unit 8

Browser route — answer them at the foot of this unit's page, with no installation: cp08-table-evidence, cp08-diagnose and cp08-what-it-shows. The browser self-check cannot run code, so it cannot grade cp08-bernoulli; it lists that checkpoint and directs you to complete it in the Python lab.

cp08-bernoulli requires a completed exercises/ex04_bernoulli.py; until you edit the stub it will report that the checkpoint has not been attempted rather than failing you.

To bring the result into the browser record, write the lab's record with python3 selfcheck.py progress --backup my-ai-literacy-record.json and load that file under "Restore from a backup" on the progress page.

To see where you have got to:

python3 selfcheck.py list
python3 selfcheck.py progress

Explore, if you want to

The oracle is there to be poked at, not only to judge you:

python3 selfcheck.py explore

Inside, b 30 prints B30 exactly, float 30 shows the exact value and the float side by side, table 20 prints a range, map shows the Note G index mapping, check <file> runs the checker, and quit leaves.

Optional CS extensions

Optional. These are outside the core study time and are not required to complete this unit or the course. Non-programmers lose no core content by skipping them.

Write the test yourself — exercises/ex05_write_a_test.py

python3 exercises/ex05_write_a_test.py

So far the checker has done the judging. Here you write it: a single function that must separate four supplied faulty implementations from one correct one, without wrongly rejecting the correct one. A test that only checks B4 == -1/30 lets one of the four straight through: the other B1 convention, which agrees at B4. The exercise grades itself against the real files in examples/ and tells you which you caught and which you missed.

Translate between Ada's indexing and ours — exercises/ex06_ada_translation.py

python3 exercises/ex06_ada_translation.py

Implement the mapping from Lovelace's Note G labels to modern indices, returning exact values and rejecting indices that have no Note G label at all. This one looks ahead to Unit 16; you can do it now or after that unit.

Marking guidance — open this once you have done the activity, to check your own work

Answer key for the Unit 8 activity. Self-check checkpoints are not answered here; they mark themselves and give their own diagnostic feedback, by either route.

Read this after attempting the activity. Where a question asks for your own routine, the key describes what a good answer looks like rather than giving one.

Before you start — the safety step

There is no single right answer, but there is a right shape of answer.

  1. Imports. fractions is expected; math is reasonable, usually for comb. Anything else needs a reason you can state. os, sys with path manipulation, socket, subprocess, urllib, requests, shutil and pathlib writes have no place in a routine that computes a number from an integer.
  2. Import-time behaviour. Anything outside a function body runs the moment the checker imports the file. A print is harmless and untidy; a file write, a network call or a subprocess is neither. Note that a if __name__ == "__main__": block does not run on import, so a demonstration loop there is fine.
  3. Unexplained code. The honest answer is often "I could not tell what this line does". That is a finding, not a failure. Code you cannot read is code you cannot vouch for, and the correct response is to remove it or not run it.

"Nothing suspicious, here is why", with specific reasons, is doing this correctly. "It looked fine" is not doing it at all.

A2 — The supplied examples

Exact figures from the checker, at the default --max-n 30:

file first divergence at n expected got diagnosis code(s)
wrong_b1_sign.py 1 -1/2 1/2 b1-sign
ada_indexing.py 1 -1/2 1/6 odd-terms, ada-indexing, off-by-one, one-observation
float_output.py 2 1/6 a float not-fraction, odd-terms
off_by_one_range.py 0 1 -1/2 odd-terms, ada-indexing, off-by-one, one-observation
correct_example.py — — — none; PASS

The float in the third row is exactly 0.16666666666666674, which is written out here rather than in the cell because a nineteen-character literal does not fit a table column on a printed page.

Notes on the details that catch people out:

wrong_b4_sign.py: first divergence at n=4, expected -1/30, got 1/30, and no named code at all. One wrong value, every other index right; the report locates it and does not name it, because the codes describe faults that recur rather than every fault there can be.

Are four codes four faults? No. ada_indexing.py makes one mistake: it files each value under Lovelace's label, so its B1, B3, B5 are her B1, B3, B5, the modern B2, B4, B6. That breaks two requirements at once -- the odd indices no longer hold zero (odd-terms) and every label names a different number from the modern one (ada-indexing). Her labels and a one-place shift then agree at every index except B0, so off-by-one matches as well, and one-observation says that the last two are one observation with two readings. B0 tells the two files apart: ada_indexing.py returns 1 there, which points at Note G's labels, and off_by_one_range.py returns -1/2, which points at a shift. A diagnostic system that reported only the first symptom would leave you fixing half the fault; one that counted codes as bugs would send you after four.

What is wrong with off_by_one_range.py? Every value it returns is a genuine Bernoulli number, and every one is filed under the wrong index: its bernoulli(0) hands back B1, its bernoulli(4) hands back B5. The shift is in the source -- bernoulli returns _b(n + 1) -- but it is easy to read past, because there is nothing badly written around it: the arithmetic is exact, the style is clean, and the values are all real. Whether n + 1 is right is a relationship between the code and a reference outside the code, and a comparison against that reference settles it at once. This is a strong argument for differential testing alongside code review.

A3 — Your own code

A good record is specific:

First divergence at n = 1, expected -1/2, got 1/2. Diagnosis: b1-sign. The routine implements the second Bernoulli convention; my prompt never said which convention I wanted, so the specification was incomplete rather than the code being wrong.

Common outcomes, and what each tells you about the prompt that produced them:

diagnosis what the prompt failed to state
b1-sign which Bernoulli convention to use
odd-terms that odd indices above 1 must return exactly zero
ada-indexing that modern indexing is required, not Note G's
off-by-one the value at named indices, anchoring where the sequence starts
not-fraction that returns must be exact fractions.Fraction, never float
raises that the function is defined for every n >= 0
no-function the required function name and signature
import-error that the module must import cleanly using only the standard library

If it passed first time, the honest answer names the clause you got for free. Typically it is exact arithmetic — a request phrased around Ada Lovelace or exact values sometimes yields Fraction without being asked, and sometimes does not. Getting it without asking is luck, and luck is not a property you can rely on next time. That observation is the whole of Unit 10 in one line.

Against your Unit 7 prediction. A good sentence names one thing your prediction got right and one thing your reading missed:

I predicted not-fraction, with the first divergence at n = 2. The report also gave b1-sign at n = 1, because I read past the line that sets the sign of B1.

A wrong prediction is a useful result, not a failed step: what you missed is what reading alone could not show you, which is why the check exists. If you wrote no prediction, say so, and do not reconstruct one now: a prediction written after the report is not a prediction. The same goes for a pass you did not predict. It tells you about the routine, not about your understanding of it (Unit 17).

A5 — what the pass does not establish. Any of these is a good answer:

B3 — Table route

The first divergence found by hand is the same index the tool would report, provided you compared in index order and stopped at the first difference. If you scanned for "the obviously wrong one" instead, you may have found a later divergence that is only a consequence of an earlier one.

The three things that are easy to miss when scanning:

  1. Odd rows. A generated table can omit the odd indices entirely, listing only B0, B1, B2, B4, B6, .... An omitted row is not the same claim as a row containing zero, and the difference matters: the specification requires bernoulli(3) to return zero, not to be undefined. If the odd rows are present but written as 0.0, ~0 or 1e-17, that is the float failure mode.
  2. Exact fractions. -0.0333... is not -1/30. It is close, and closeness is the property the specification rules out. Any decimal at all is a not-fraction finding.
  3. B30. A table that agrees perfectly from B0 to B12 and then gives 601580873.9 or 6.0158e8 for B30 has told you exactly what you needed to know. If B30 was refused or omitted, treat that as unverified rather than as correct.

Against your Unit 7 prediction. The guidance under A3 applies here too: name one thing your prediction got right and one thing your reading missed, and if you wrote no prediction, say so rather than reconstructing one. Compare the rows, not your memory of them: "I expected a decimal at index 4 and a missing row at index 3" can be checked against your table, and "I thought it might be wrong" cannot.

B5 — why agreement does not establish that a program produced the values. A table in a chat window is generated text. It may have been produced by a routine, copied from a published table, or generated the same way as any other text. Those are different claims with different reliability, and the table looks identical in all three cases. The values being right tells you the values are right; it tells you nothing about the existence or behaviour of a program. If your task requires a working routine, that is a separate claim needing separate evidence — which is what Route A supplies.

The two routes

Both routes evidence the same skill: ordered comparison against a trusted reference, stopping at the first divergence, and naming the fault. If you completed Route B you have verified. Route A is not the real version and Route B the substitute.

What makes the work good, by either route, is specificity: an index, a pair of values, a named failure mode, and a sentence about which requirement was missing. "It was wrong and then I fixed it" is not a verification record.

Optional CS extensions

These are optional and outside the core study time. Do not treat them as required.

ex05_write_a_test.py. The exercise grades itself, so no key is given here. The instructive failure is a test that catches all four faulty implementations by being aggressive and then wrongly rejects correct_example.py — the file reports that separately, and says why it is worse than missing a fault: a test that fails good code is a test people learn to ignore. The second instructive failure is a test built from == against a handful of known values, which lets the type-related and range-related faults through. A test that works generally checks a property and a type, not only a list of values.

The point to take from this: every clause you needed in your test corresponds to a clause the specification should have contained. That list is the one you build in Unit 10.

ex06_ada_translation.py. Also self-grading. Note that the exercise requires rejecting Ada B2 and Ada B0 with a ValueError, not only computing Ada B7 correctly. Learners often implement the mapping and forget the rejection, and the exercise says why that matters: a translation layer that silently accepts nonsense from the other side is how convention bugs travel between systems.

Checkpoints

cp08-bernoulli, cp08-table-evidence, cp08-diagnose and cp08-what-it-shows are self-marking, with their own diagnostic feedback, and their answers are deliberately not reproduced here. You reach cp08-diagnose and cp08-what-it-shows by either route; cp08-bernoulli belongs to the Python route and cp08-table-evidence to the browser/table route:

python3 selfcheck.py run cp08-diagnose  # or in the browser
python3 selfcheck.py run cp08-what-it-shows  # or in the browser
python3 selfcheck.py run cp08-bernoulli    # Python route only
python3 selfcheck.py run cp08-table-evidence  # table route

Your private activity record

Browser storage is not a permanent copy

Progress is kept only in this browser, profile and device. Private browsing, clearing site data, removing the profile, a browser reset, storage eviction or device loss can erase it. Keep important answers and contributions separately.

These notes stay in this browser unless you download a backup or activity log.