The verification lab and self-check

This directory holds the tools you use to check your own work: a trusted reference implementation of the Bernoulli numbers, a differential checker you can point at any Python file, and a self-check you can run repeatedly as you work through the units.

Two routes exist, and they are equals. The browser route (build/site/selfcheck.html) needs no installation and covers the same checkpoints. The Python route described here suits you if you already run Python or want to see the checker work on real code. Choose on preference and circumstance, not on confidence. Nothing in the core module depends on installing anything.

Separately from both, the table-based verification path in Units 3, 5 and 7 lets you check generated output against a small table you fill in by hand. That is a first-class method — it is how the oracle itself was validated against published values — not a substitute for anything.

Requirements

Python 3 and nothing else. Every file here uses only the standard library (fractions, math, argparse, json, importlib, subprocess, pathlib). There is no pip install step, no virtual environment, and no network access at any point.

Verified on Python 3.13.14. Anything from Python 3.9 onwards should work.

On Windows, use py or python wherever these pages say python3.

Run all commands from inside course/lab, because the example and exercise paths in these commands are relative to that folder:

cd course/lab

Safety: read before you run

check.py imports and executes the file you hand it. So does selfcheck.py when it runs a code checkpoint for you. There is no sandbox.

AI-generated code is code from an untrusted source. It is not malicious by design, but it was not written by anyone who is answerable for it, and you cannot tell what it does by looking at how confident it sounds. Treat it exactly as you would treat a script downloaded from a stranger:

• Read the whole file first. Look for anything that touches the filesystem, the network, or the shell. A Bernoulli routine needs none of those.
• Prefer a disposable environment: a scratch directory, a container, a throwaway virtual machine, or a machine with nothing on it you would mind losing. Not your only copy of your coursework.
• Do not paste confidential, personal, unpublished, or assessment-restricted material into an AI system in order to get the code in the first place.
• Notice that you had to make this decision. That noticing is part of the module, not an obstacle to it.

What each file is

The example implementations

The exercises

The optional CS extensions are labelled optional in the module because they are optional. Skipping them costs you no core content.

The interface contract

This is the contract check.py enforces, and the same words appear in the Unit 6 precise prompt:

bernoulli(n: int) -> fractions.Fraction

Defined for every n >= 0; first-Bernoulli convention B1 = -1/2; exact Fraction returns only, never float; returns Fraction(0) for odd n > 1.

That is the whole specification. It is short enough to read and precise enough to test against — and a specification precise enough to test against is a specification precise enough to prompt with.

Failure-mode codes

These are the names check.py emits. They are the module's fixed failure-mode codes, and the module uses these names and no synonyms.

check.py also prints three structural markers alongside these: divergence gives the first index where your values and the oracle's disagree, no-file means the path does not exist, and not-callable means bernoulli exists but is not a function. They locate the problem; the eight codes above name it.

One report often carries several codes. examples/off_by_one_range.py produces divergence, odd-terms and off-by-one together, because a one-index shift also drags non-zero values onto the odd indices. Read the codes as a description of the shape of the fault, not as a count of separate bugs.

Commands

Every transcript below is real output from running the command in this directory. Where a command is interactive, what you type is shown after the prompt.

Print the oracle's table

$ python3 reference_bernoulli.py
Bernoulli oracle - convention: first Bernoulli numbers (B_n^-), B1 = -1/2

  n   B_n
  --  ---------------------------
   0  1
   1  -1/2
   2  1/6
   3  0
   4  -1/30
   5  0
   6  1/42
   7  0
   8  -1/30
   9  0
  10  5/66
  11  0
  12  -691/2730

Larger index, where float implementations fail:
  B30 = 8615841276005/14322

Note G indexing (Lovelace's label -> modern label -> value):
  Ada B1  ->  modern B2  =  1/6
  Ada B3  ->  modern B4  =  -1/30
  Ada B5  ->  modern B6  =  1/42
  Ada B7  ->  modern B8  =  -1/30

OK: oracle agrees with all 22 published values, odd terms are zero.

Exit status 0. If the oracle ever disagreed with its published table it would print FAIL: with the offending indices and exit 1. Start here: this is the table you check everything else against, and it is short enough to copy onto paper for the table-based route.

Check a file against the oracle

$ python3 check.py examples/wrong_b1_sign.py
====================================================================
Checking: examples/wrong_b1_sign.py
Oracle convention: first Bernoulli numbers (B_n^-), B1 = -1/2
====================================================================

FAIL - 2 problem(s) found.

[divergence] First divergence at n=1: expected -1/2, got 1/2.
    Everything below this index matched. The first divergence is the
    best place to start debugging. It does not prove that later
    differences have the same cause.
    See: Unit 5 (differential testing against an oracle)

[b1-sign] B1 has the wrong sign for this module's convention.
    Your routine returns B1 = +1/2. That is the *second* Bernoulli
    convention, and it is not wrong mathematically -- but this module
    fixes the *first* convention, B1 = -1/2. The conventions differ
    only at B1, but leaving that choice unstated is still a
    specification failure, not an arithmetic failure.
    See: Unit 3 (the convention trap), Unit 6 (state the convention in the prompt)

Fix the specification or the code, then run this check again.

Exit status 1 on failure, 0 on success. A passing run looks like this:

$ python3 check.py examples/correct_example.py --verbose
====================================================================
Checking: examples/correct_example.py
Oracle convention: first Bernoulli numbers (B_n^-), B1 = -1/2
====================================================================

PASS - agrees with the oracle for every n from 0 to 30,
       returns exact Fractions, and odd terms above B1 are zero.

Passing is not the end of the job. You still have to be able to say
*why* it is correct. That is the Unit 8 defence.

(Indices compared: 0 to 30)

Options:

Point it at your own file the same way: python3 check.py my_bernoulli.py. Read the file first — see the safety section above.

List the checkpoints

$ python3 selfcheck.py list
Self-check checkpoints
----------------------------------------------------------------------

Unit 0
  [ ] cp00-contract        (choice)    The learning contract

Unit 1
  [ ] cp01-ordered-steps   (multi)     What the procedure left unsaid

Unit 2
  [ ] cp02-risk-triage     (match)     Triage by how it can be wrong
  [ ] cp02-system-layers   (match)     Separate the assistant's system layers

Unit 3
  [ ] cp03-b4-by-hand      (fraction)  B4 by hand
  [ ] cp03-convention      (choice)    Which convention is this?
  [ ] cp03-ada-index       (integer)   Reading Ada's index

Unit 4
  [ ] cp04-trust-triage    (choice)    What you can trust before testing

Unit 5
  [ ] cp05-bernoulli       (code)      Make the checker pass
  [ ] cp05-table-evidence  (match)     State what a table comparison establishes
  [ ] cp05-diagnose        (choice)    Name the failure mode

Unit 6
  [ ] cp06-spec-clauses    (match)     Which clause kills which bug

Unit 7
  [ ] cp07-note-g          (fraction)  Comparing against the historical table

Unit 8
  [ ] cp08-disclosure      (multi)     What a defence has to contain

Unit 9
  [ ] cp09-sort-the-claims (match)     Sort the claims

Run one with:  python3 selfcheck.py run <id>

[x] marks a checkpoint you have passed. The kind in brackets tells you what sort of answer is expected: a fraction such as -1/30, a whole number, one option number, several option numbers, an ordered list of option numbers, or — for code — a file you edit.

Run one checkpoint

$ python3 selfcheck.py run cp03-ada-index
----------------------------------------------------------------------
cp03-ada-index  (Unit 3)  -  Reading Ada's index
----------------------------------------------------------------------
Note G computes the number Lovelace calls B7. In modern notation,
where every index is numbered including the zero ones, which B_n is
that? Enter the number only.

Answer with a whole number.

Your answer (or 'skip'): 8

PASS
Ada B1/B3/B5/B7 are the modern B2/B4/B6/B8, so Note G's worked example
produces the modern B8 = -1/30. A historical notation that means
something different from the identical modern notation is a
specification hazard, and it is exactly the kind of thing a language
model will blur together.

A wrong answer that matches a known misconception gets that misconception named rather than a bare "incorrect":

$ python3 selfcheck.py run cp03-b4-by-hand
----------------------------------------------------------------------
cp03-b4-by-hand  (Unit 3)  -  B4 by hand
----------------------------------------------------------------------
Using the recurrence sum_{j=0}^{n} C(n+1, j) * B_j = 0 with B0 = 1 and
the first-Bernoulli convention B1 = -1/2, work out B4 by hand. Enter
it as an exact fraction, for example -1/30 or 5/66.

Answer as an exact fraction, e.g. -1/30

Your answer (or 'skip'): 1/30

Not yet.
Right magnitude, wrong sign. Check the sign of the term you moved
across the equals sign when you solved for B4 -- the whole sum equals
zero, so the highest term takes the sign of everything else negated.

You get three attempts per session, and typing skip (or pressing Enter on an empty line) moves on. On checkpoints where deriving the value is the point, such as cp03-b4-by-hand, a failure never reveals the answer. It gives you the next step instead.

The code checkpoint delegates to check.py, and recognises an untouched stub rather than reporting a crash:

$ python3 selfcheck.py run cp05-bernoulli
----------------------------------------------------------------------
cp05-bernoulli  (Unit 5)  -  Make the checker pass
----------------------------------------------------------------------
Complete course/lab/exercises/ex04_bernoulli.py so that
bernoulli(n) meets the contract, then run: python3 selfcheck.py run
cp05-bernoulli

Not attempted yet - the stub is still unedited.
Open exercises/ex04_bernoulli.py and complete it, then run this again.

run also accepts all (or no target at all) to work through every checkpoint in order, and --answer TEXT to supply a single answer without being prompted.

Run every checkpoint in a unit

$ python3 selfcheck.py run --unit 3
----------------------------------------------------------------------
cp03-b4-by-hand  (Unit 3)  -  B4 by hand
----------------------------------------------------------------------
Using the recurrence sum_{j=0}^{n} C(n+1, j) * B_j = 0 with B0 = 1 and
the first-Bernoulli convention B1 = -1/2, work out B4 by hand. Enter
it as an exact fraction, for example -1/30 or 5/66.

Answer as an exact fraction, e.g. -1/30

Your answer (or 'skip'): -1/30

PASS
You now know what the answer *should* be before any AI produced it.
Deriving the value independently gives you one strong reference for
judging generated output; an authoritative table or separately
implemented method can also provide independent evidence. This value
catches the sign-convention bug in Unit 5.

----------------------------------------------------------------------
cp03-convention  (Unit 3)  -  Which convention is this?
----------------------------------------------------------------------
A program prints B0 = 1, B1 = 1/2, B2 = 1/6, B4 = -1/30. Its author
says it is correct and they are not lying. What is going on?

  (1) It uses the second Bernoulli convention (B1 = +1/2); every other
value agrees.
  (2) It has a sign bug that happens to affect only B1.
  (3) It is using Lovelace's Note G indexing.
  (4) It is correct and this module's oracle is wrong.

Answer with one option number.

Your answer (or 'skip'): 1

PASS
Both conventions are standard and published. Neither program is
broken; the *specification* was incomplete. This is why the module's
contract states B1 = -1/2 explicitly, and why Unit 6 puts that
sentence in the prompt.

----------------------------------------------------------------------
cp03-ada-index  (Unit 3)  -  Reading Ada's index
----------------------------------------------------------------------
Note G computes the number Lovelace calls B7. In modern notation,
where every index is numbered including the zero ones, which B_n is
that? Enter the number only.

Answer with a whole number.

Your answer (or 'skip'): 8

PASS
Ada B1/B3/B5/B7 are the modern B2/B4/B6/B8, so Note G's worked example
produces the modern B8 = -1/30. A historical notation that means
something different from the identical modern notation is a
specification hazard, and it is exactly the kind of thing a language
model will blur together.

----------------------------------------------------------------------
Session: 3 of 3 checkpoints passed.
----------------------------------------------------------------------

See how far you have got

$ python3 selfcheck.py progress
Passed 7 of 15 checkpoints.

  Unit 0  [#]  1/1
  Unit 1  [#]  1/1
  Unit 2  [##]  2/2
  Unit 3  [###]  3/3
  Unit 4  [.]  0/1
  Unit 5  [...]  0/3
  Unit 6  [.]  0/1
  Unit 7  [.]  0/1
  Unit 8  [.]  0/1
  Unit 9  [.]  0/1

Next up: cp04-trust-triage (Unit 4)

Export a record for your evidence pack

$ python3 selfcheck.py progress --evidence
# Self-check record

Generated 2026-07-24 00:23 UTC.

Passed 7 of 15 checkpoints.

| checkpoint | unit | passed | attempts |
|---|---|---|---|
| cp00-contract | 0 | yes | 1 |
| cp01-ordered-steps | 1 | yes | 1 |
| cp02-risk-triage | 2 | yes | 1 |
| cp02-system-layers | 2 | yes | 1 |
| cp03-b4-by-hand | 3 | yes | 1 |
| cp03-convention | 3 | yes | 1 |
| cp03-ada-index | 3 | yes | 1 |
| cp04-trust-triage | 4 | no | 0 |
| cp05-bernoulli | 5 | no | 0 |
| cp05-table-evidence | 5 | no | 0 |
| cp05-diagnose | 5 | no | 0 |
| cp06-spec-clauses | 6 | no | 0 |
| cp07-note-g | 7 | no | 0 |
| cp08-disclosure | 8 | no | 0 |
| cp09-sort-the-claims | 9 | no | 0 |

This record shows which checkpoints were passed. It is not a claim that
the learner can defend the answers; that is what the Unit 8 defence is for.

That output is Markdown, so you can redirect it straight into your evidence pack:

$ python3 selfcheck.py progress --evidence > my-selfcheck-record.md

Nothing sends it anywhere. Exporting it is a decision you make.

To move your checkpoint results into the browser course, or to keep a restorable backup, write the portable JSON record:

$ python3 selfcheck.py progress --backup my-ai-literacy-record.json
Portable learner record written to my-ai-literacy-record.json.

The browser progress page imports the same format. The readable Markdown evidence record is for you or an instructor to read; the JSON file is the format that can restore progress.

Explore the oracle

$ python3 selfcheck.py explore
Bernoulli oracle - exploration mode
Convention: first Bernoulli numbers (B_n^-), B1 = -1/2

Commands:
  b <n>            value of the modern B_n            e.g. b 30
  ada <n>          value of Lovelace's B<n>           e.g. ada 7
  map              the Note G index mapping table
  table [n]        modern B_0 .. B_n (default 12)
  float <n>        B_n as an exact fraction and as a float, side by side
  check <file>     run the differential checker on a file
  help             this list
  quit             leave

explore> b 4
  B4 = -1/30
explore> ada 7
  Ada B7 = modern B8 = -1/30
explore> map
  Ada B1  ->  modern B2  =  1/6
  Ada B3  ->  modern B4  =  -1/30
  Ada B5  ->  modern B6  =  1/42
  Ada B7  ->  modern B8  =  -1/30
explore> float 30
  exact  B30 = 8615841276005/14322
  float  B30 = 601580873.9006424
  -> the float is NOT equal to the exact value.
explore> table 6
  B0   = 1
  B1   = -1/2
  B2   = 1/6
  B3   = 0
  B4   = -1/30
  B5   = 0
  B6   = 1/42
explore> quit

This is where the module stops being a reading and starts being a laboratory. Look up any value you want to check by hand, watch float 30 fail, and run check <file> on anything you have generated without leaving the session.

Clear your progress

$ python3 selfcheck.py reset
Progress cleared.

Running it again when there is nothing to clear:

$ python3 selfcheck.py reset
No progress to clear.

Write the test yourself (optional CS extension)

The stub reports that it is a stub:

$ python3 exercises/ex05_write_a_test.py
Write your test here, then run this file.

Once you have filled in catches_the_bug, the exercise grades your test against the real example implementations. A test that only checks B4 == -1/30 looks reasonable and lets two of the four faults straight through:

$ python3 exercises/ex05_write_a_test.py
Running your test against every example implementation.
------------------------------------------------------------------
  MISSED wrong_b1_sign        (the other B1 convention)
  MISSED wrong_odd_terms      (Note G non-zero-only indexing)
  caught float_output         (floating point, not exact)
  caught off_by_one_range     (index shifted by one)
  passed correct_example      (a correct implementation)
------------------------------------------------------------------
Not yet.
  Slipped through: wrong_b1_sign, wrong_odd_terms
  Ask what property each of those violates that a correct one does not.

A test that checks the type, the convention, the zero odd terms and a large index separates all four:

$ python3 exercises/ex05_write_a_test.py
Running your test against every example implementation.
------------------------------------------------------------------
  caught wrong_b1_sign        (the other B1 convention)
  caught wrong_odd_terms      (Note G non-zero-only indexing)
  caught float_output         (floating point, not exact)
  caught off_by_one_range     (index shifted by one)
  passed correct_example      (a correct implementation)
------------------------------------------------------------------
PASS - your test separates all four faults from the correct version.

Now notice what you had to know in order to write it. Every clause in
your test corresponds to a clause the specification should have had.
That is the same list you build in Unit 6.

Note that wrongly rejecting correct_example is reported as an error too. A test that fails good code is worse than no test, because you learn to ignore it.

Check the framework itself

The lab's own test suite runs with pytest if you have it, and with a standard-library fallback runner if you do not:

$ python3 tests/run_tests.py
...
==================================================================
81 passed, 0 failed, 6 skipped
==================================================================

The counts change as the suite grows. A skipped test is one this plain runner cannot drive (it needs pytest fixtures, and runs under pytest instead); it is not a failure. You are not required to run this — it exists so that the tools judging your work have themselves been checked against published values and known-bad examples.

What is recorded, and what is not

• The self-check is formative and unmarked unless your instructor says otherwise. It exists so you can find out what you have understood while there is still time to do something about it.
• Progress is stored locally, in .selfcheck-progress.json in this directory. It uses the portable learner-record format documented in course/learner-record-schema.json and records checkpoint answers, pass state, attempts, and timestamps. Keep a separate backup if the work matters.
• Nothing is uploaded. There is no network code anywhere in this directory, no analytics, and no account. Delete the file, or run selfcheck.py reset, and the record is gone.
• If your instructor asks for evidence, you produce it yourself with progress --evidence and hand it over deliberately. See course/learner-evidence-template.md.
• The answers are in each unit's activity.md (which checkpoints.py reads), in plain sight. This is a self-check, not an exam. Reading them costs you the only thing the checkpoint was offering.

The browser route

build/site/selfcheck.html runs the same 15 checkpoints, with the same ids, the same expected answers and the same diagnostic wording, in a supported browser. It is generated from the same activity data by scripts/build_selfcheck.py, which is why the two routes cannot drift apart. It is self-contained, makes no network requests, and can work from a file:// URL. Browser storage behaviour for local files varies, so use the published course site for durable local progress and keep a portable backup either way.

Use it if you would rather not install or run Python, if you are working on a machine where you cannot, or simply because you prefer it. It is not a reduced version of the Python route, and choosing it does not put you on a lesser path through the module. What it does not include is the checker running against real code — for that, either use the Python route or follow the table-based verification worksheet in Unit 5, which reaches the same conclusion by hand.

Where this fits

Related material: course/course-map.md for how the units map to the wider course, course/glossary.md for the terms used here, course/misconceptions.md for the errors these tools are built to catch, course/historical-notes.md for Note G and its citations, and course/references.md for sources.