Glossary

Terms used across this module, with the unit that introduces each one. Unit titles and numbers come from course/course-map.md; this file does not restate them.

Where a term has a loose everyday meaning and a precise one here, the precise one is what the module means.

Index

Definitions

algorithm

A finite, unambiguous description of how to compute something: a fixed set of steps that, applied to valid input, produces the intended output and stops. Every step must be executable without judgement or interpretation by whatever carries it out.

Introduced in Unit 1. Note G is the module's example: a computation written out in enough detail that a machine with no knowledge of Bernoulli numbers could carry it out.

ordered procedure

The plain-language name this module uses for an algorithm written as a numbered sequence of operations, each one acting on named quantities in a stated order. The emphasis is on ordered: the steps are not a list of things to do, but a sequence in which position carries meaning, because each step depends on the result of the ones before it.

Introduced in Unit 1.

determinism

The property that the same input, run through the same procedure, always produces the same output. A deterministic procedure has no dependence on chance, on timing, or on anything not written down. reference_bernoulli.bernoulli(4) returns -1/30 today, tomorrow, and on any machine.

Introduced in Unit 1. Contrast with a language model, whose output for the same prompt may differ between runs.

state

The set of values a procedure is currently holding, and which it may change as it proceeds. In Note G these are the engine's numbered variables; in the lab oracle they are the partial sums accumulated inside the recurrence. Tracking state is what lets you say precisely where a computation has got to, and therefore where it went wrong.

Introduced in Unit 1.

language model

A statistical model of sequences of text, fitted to a large body of existing text, which assigns probabilities to what token might come next given what has come before. Text is generated by sampling from those probabilities repeatedly.

A language model is a description of regularities in text, not a store of verified facts and not an agent with intentions. Describing what it produces is accurate; describing what it "believes" or "wants" is not.

Introduced in Unit 2.

next-token prediction

The mechanism by which a language model produces text: given the sequence so far, it produces a probability distribution over possible next tokens, one is selected, it is appended, and the process repeats. A token is a short chunk of text — often a word or part of a word.

Two consequences matter for this module. First, the base generation objective is a likely continuation, not a direct correctness test. Second, correct and incorrect output can both be fluent, so fluency alone does not establish correctness.

Introduced in Unit 2.

attention

The mechanism inside a transformer language model that lets the representation at each position combine information from relevant positions elsewhere in the available context. It is a way of combining context; it is not a database lookup, a citation, or a truth test.

Introduced in Unit 2.

pre-training

The first stage of building a language model: adjusting its many numerical parameters to improve next-token prediction over a large training corpus. The result is the base model — a description of regularities in that text, not a store of verified facts.

Introduced in Unit 2.

post-training

Further training applied to a base model — on demonstrations and preference judgements — so a deployed assistant follows instructions and behaves as its makers intend. It changes how the model responds; it does not make a generated claim correct by definition.

Introduced in Unit 2.

retrieval

A system layer some deployed assistants add: fetching documents or search results and placing them in the model's context before it generates. Retrieval can ground an answer in a real source, but the generated text still has to be checked against that source — attaching a document is not the same as citing it correctly.

Introduced in Unit 2.

plausibility

The property of reading as though it could be right: well-formed, idiomatic, consistent in tone, and shaped like a correct answer. Language-model training and generation strongly reward plausible continuation.

Plausibility is not correctness, and — importantly — it is not evidence of correctness either. A plausible wrong answer can look as polished as a plausible right answer, which is why the module builds a separate means of checking.

Introduced in Unit 2.

hallucination

The established term for output from a generative model that is fluent, well-formed, and false: an invented citation, a non-existent function, a confident date that no source supports.

The word is a metaphor and should not be taken literally. The model is not perceiving anything, not mistaken about anything, and not doing something different from what it does when it is right. It is producing a statistically plausible continuation in both cases; "hallucination" simply names the subset of those continuations that do not correspond to the relevant source, specification, or world. Surface reading may expose some errors, but it cannot establish the claim; that requires appropriate evidence.

Introduced in Unit 2. Used consistently in this sense throughout the module.

independent verification

A check whose evidential basis is separable from the assertion being checked: for example, an authoritative source, an exact calculation, a test oracle built by a different method, or accountable review against explicit criteria.

Re-prompting or self-critique can improve an answer [T9], but the same system may preserve the same assumptions. Improvement is useful; it is not the same claim as independent verification.

Introduced in Unit 2; practised in Unit 5.

convention

A choice about notation or definition that is agreed rather than derived, where a different choice would be equally valid and would give different results. The Bernoulli numbers have two published conventions: the first Bernoulli numbers, with B1 = -1/2, and the second, with B1 = +1/2. Every other value is identical.

Conventions cause a distinctive kind of fault. Two programs can both be correct and defensible while disagreeing at B1, because the specification never said which convention was meant. They agree at every other index, but the unresolved meaning of B1 remains a specification defect. This module fixes the first convention, B1 = -1/2, and states it in the contract.

Introduced in Unit 3. Failure-mode code: b1-sign.

exact rational arithmetic

Arithmetic on fractions represented as an exact numerator and denominator, with no rounding at any point. Python's fractions.Fraction does this: Fraction(1,3) is one third, not an approximation to it.

The lab oracle uses exact rational arithmetic because Bernoulli numbers have large numerators and denominators — B30 = 8615841276005/14322 — and comparing generated output against a reference is only meaningful if both sides are exact. An approximate comparison has to choose a tolerance, and any tolerance loose enough to pass rounding error is loose enough to pass a real bug.

Introduced in Unit 3.

floating point

The binary approximation to real numbers used by default in most programming languages, including Python's float. It is fast, fixed in size, and inexact: most fractions cannot be represented, so results carry small errors that accumulate.

For small Bernoulli indices, floating-point output looks convincing — -0.03333... really is close to -1/30. By B30 the error is unmistakable. Producing floats where exact values were specified is a failure of the specification's arithmetic requirement, not a rounding detail to be tolerated.

Introduced in Unit 3. Failure-mode code: not-fraction.

ground truth

The independently established answer against which a result is judged. In this module, ground truth for the Bernoulli numbers comes from published tables and from the value you derive yourself by hand in Unit 3 — not from the program being tested, and not from a second generated answer that agrees with the first.

The ordering matters: ground truth has to be established before you look at the output you are judging, or you will find yourself deciding that whatever you got must have been right.

Introduced in Unit 3.

off-by-one

An indexing error in which every value is correct but shifted by one position, so that item n returns what belongs at n+1 or n-1. It is the archetypal programming mistake because it survives inspection: the arithmetic is right, the values are all genuine, and nothing looks wrong until you compare index by index against a reference.

Introduced in Unit 4. Failure-mode code: off-by-one.

verification

Establishing that a result is correct by checking it against something independent of the process that produced it. Reading the output again, asking the model whether it is sure, or noting that it looks reasonable are none of them verification, because none of them is independent.

In this module verification takes three concrete forms, all equally valid: comparing against a table you filled in by hand, comparing against a published source, and comparing against a test oracle by running code.

Named in Unit 0 as part of the learning contract; practised throughout and made central in Unit 5.

test oracle

A trusted source of correct answers, used to judge a piece of code under test. course/lab/reference_bernoulli.py is this module's oracle.

An oracle is only worth as much as its own validation, so this one is checked in two directions: against 22 values transcribed from published tables, which it did not compute, and against properties that must hold regardless of how it computes anything.

Introduced in Unit 5.

differential testing

Running two implementations of the same specification on the same inputs and comparing the results, on the principle that two independent routes to the same answer are much stronger evidence than one route agreeing with itself. course/lab/check.py does this: your routine against the oracle, index by index.

The independence is what carries the weight. examples/correct_example.py deliberately uses a different algorithm from the oracle — the Akiyama–Tanigawa transform rather than the binomial recurrence — so that agreement is informative rather than circular.

Introduced in Unit 5.

property-based check

A check that tests a property which must hold for every valid input, rather than comparing specific input-output pairs. The module's example: every odd Bernoulli number above B1 is exactly zero, so B_(2k+1) = 0 for all k >= 1.

Property checks catch faults that a table of examples misses, because they do not depend on having chosen the right examples. They also state, in code, something a reader would otherwise have to know.

Introduced in Unit 5. Failure-mode code: odd-terms.

first divergence

The lowest index at which a routine under test and the oracle disagree. The checker reports this specifically because it gives a reproducible boundary: every lower tested input matched. It is often a useful place to start debugging, but it does not prove that later failures share the same cause.

Introduced in Unit 5.

overreliance

Depending on a tool beyond the point where you can tell whether its output is right. It is not the same as using AI heavily; it is using AI in place of the understanding that would let you check it.

The practical test is whether you could detect an error if there were one. If a generated answer would look identical to you whether it was correct or not, you are relying on it rather than using it. This is why the module has you derive B4 by hand before any code is generated: the understanding has to come first, or there is nothing to check against.

Introduced in Unit 6.

established, contested, unsourced

The module's three buckets for the status of a claim. Established: you can name where it comes from, and the source is one that you, or someone accountable to you, has actually looked at. Contested: sources disagree, or a single account has been copied forward through many retellings without an independent check behind it. Unsourced: you cannot currently say where the claim came from — which is not the same as false.

A claim's bucket is part of the claim, and a claim moves buckets when a check is actually done, not when it is repeated more.

Introduced in Unit 7, §6 of its reading; the same status labels are used throughout course/references.md.

disclosure

A short, factual statement of how AI was used in producing a piece of work: what was generated, at what stage, and what was done with it afterwards. Disclosure answers the question how was this made.

It is a statement about provenance. It is not a claim about quality, and it does not remove the user's responsibility for the output. Other people and organisations may also have duties in a wider system.

Named in Unit 0; templated and practised in Unit 8. See course/learner-evidence-template.md.

defence

A short statement of why the work is correct, citing the checks that were run and the reasoning that was followed. Defence answers the question why is this right.

A defence rests on evidence: a value derived by hand, a table compared, a checker run and its output. It never rests on the authority of the tool that produced the work. "The model was confident" and "the code looked right" are not defences.

Disclosure and defence are separate statements answering separate questions, and the module asks for both. The gap between them is the gap between "I generated this" and "I understand and can defend this".

Introduced in Unit 8.

provenance

Where a thing came from: who or what produced it, by what process, at what stage, and what happened to it afterwards. A disclosure is a statement of provenance. Provenance is worth recording honestly, and it is not evidence of correctness — knowing how a piece of work was made tells you what to check, not whether it is right.

Introduced in Unit 8.

warrant

What entitles you to assert a claim: evidence you can show — a value derived by hand, a comparison recorded, a checker's output — together with the specification or assumption the evidence was gathered against. Warrant is about what you can point at, not about who produced the text. A defence carries warrant; a disclosure carries provenance; neither substitutes for the other.

Introduced in Unit 8; taken up again in Unit 9, where a warranted claim carries its index — who observed, in what context, with what record.

first-order and second-order reasoning

In Unit 9, first-order reasoning describes an observed system from the outside; second-order reasoning includes the observer in the system being described, so that claims carry a three-part index: who observed, in what context, with what record.

This use of "second-order" comes from the second-order cybernetics tradition [C1], developed from the Macy Conferences onwards. It is not the distinction between first-order and second-order logic, and it is not presented as settled terminology across every field.

Introduced in Unit 9.

observer

In Unit 9's sense, whoever or whatever is doing the describing or the checking. The observer question is: when you check something, are you outside the system you are checking, or part of it? Including the observer makes a claim more checkable, not less, because an indexed claim — who observed, in what context, with what record — says what would have to be compared with what. Maturana's compact version: anything said is said by an observer [C3, p. 8].

Introduced in Unit 9.

Macy Conferences

Ten meetings on cybernetics held between 1946 and 1953, funded by the Josiah Macy Jr. Foundation and chaired by Warren McCulloch [C4], with deliberately mixed participants — among them Wiener, von Neumann, Shannon, Bateson, Mead and von Foerster. The reflexive question behind second-order cybernetics surfaced there and was developed afterwards. A recognised research tradition with a literature — and a tradition rather than a theorem.

Introduced in Unit 9.

understanding

This module uses a working structural account, not a settled definition: your internal model of a target — another agent, a process, a piece of code, or a theory — supports understanding when its parts, relationships, and predictions continue to correspond reliably with the target under relevant checks. The verification lab gathers evidence for that correspondence through input–output agreement, property checks, and the first divergence. It does not measure understanding directly or completely.

Understanding is separate from correctness. It is a structural property — how well your model maps the target — not a claim that the target is right. You can understand a flawed framework perfectly: the module's authors mapped the as-printed Note G table and predicted what it computes without that value being correct. The capacity to map (understanding) and the empirical accuracy of the map (correctness) are two different things, and a defence needs both.

For the formal statement — understanding as a structure-preserving map f : M1 → M2 whose validity is measured by consistency, and its decoupling from truth — see course/second-order.md. Causal accounts emphasise explaining interventions; counterfactual accounts emphasise predicting what would change under different conditions. They are alternatives and possible additions; this module uses the structural account without claiming to settle the question.

Introduced in Unit 8; formalised through the second-order material in Unit 9.

formative and summative

Formative work exists to tell you — and possibly your instructor — what you have understood while there is still time to act on it. It is not marked, or is marked only for feedback.

Summative work is assessed and contributes to a result.

The self-check in course/lab/ and the browser route are formative and unmarked unless your instructor states otherwise. The distinction matters for how you should use them: on formative work, discovering that you were wrong is the useful outcome, and a checkpoint you failed twice before passing has taught you more than one you passed first time.

Introduced in Unit 0.