Glossary

Terms used across this module, with the unit that introduces each one. The units restate them where they are first needed.

Where a term has a loose everyday meaning and a precise one here, the precise one is what the module means.

Index

Definitions

algorithm

A finite, unambiguous description of how to compute something: a fixed set of steps that, applied to valid input, produces the intended output and stops. Every step must be executable without judgement or interpretation by whatever carries it out.

Introduced in Unit 2. Note G is the module's example: a computation written out in enough detail that a machine with no knowledge of Bernoulli numbers could carry it out.

ordered procedure

The plain-language name this module uses for an algorithm written as a numbered sequence of operations, each one acting on named quantities in a stated order. The emphasis is on ordered: the steps are not a list of things to do, but a sequence in which position carries meaning, because each step depends on the result of the ones before it.

Introduced in Unit 2.

determinism

The property that the same input, run through the same procedure, always produces the same output. A deterministic procedure has no dependence on chance, on timing, or on anything not written down. reference_bernoulli.bernoulli(4) returns -1/30 today, tomorrow, and on any machine.

Determinism is a property of the procedure as specified. A real implementation of it can still vary, because it runs somewhere: a search returns what the index holds today, a file operation depends on what is on disk and who may write to it, and a network call can time out. Unit 11 turns on this distinction — the tool an agent calls is not the safe step simply because the procedure it implements is deterministic.

Introduced in Unit 2. Contrast with a language model, whose output for the same prompt may differ between runs.

state

The set of values a procedure is currently holding, and which it may change as it proceeds. In Note G these are the engine's numbered variables; in the lab oracle they are the partial sums accumulated inside the recurrence. Tracking state is what lets you say precisely where a computation has got to, and therefore where it went wrong.

Introduced in Unit 2.

language model

A statistical model of sequences of text, fitted to a large body of existing text, which assigns probabilities to what token might come next given what has come before. Text is generated by sampling from those probabilities repeatedly.

A language model is a description of regularities in text, not a store of verified facts and not an agent with intentions. Describing what it produces is accurate; describing what it "believes" or "wants" is not.

Introduced in Unit 3.

next-token prediction

The mechanism by which a language model produces text: given the sequence so far, it produces a probability distribution over possible next tokens, one is selected, it is appended, and the process repeats. A token is a short chunk of text — often a word or part of a word.

Two consequences matter for this module. First, the base generation objective is a likely continuation, not a direct correctness test. Second, correct and incorrect output can both be fluent, so fluency alone does not establish correctness.

Introduced in Unit 3.

attention

The mechanism inside a transformer language model that lets the representation at each position combine information from relevant positions elsewhere in the available context. It is a way of combining context; it is not a database lookup, a citation, or a truth test.

Introduced in Unit 3.

pre-training

The first stage of building a language model: adjusting its many numerical parameters to improve next-token prediction over a large training corpus. The result is the base model — a description of regularities in that text, not a store of verified facts.

Introduced in Unit 3.

post-training

Further training applied to a base model — on demonstrations and preference judgements — so a deployed assistant follows instructions and behaves as its makers intend. It changes how the model responds; it does not make a generated claim correct by definition.

Introduced in Unit 3.

retrieval

A system layer some deployed assistants add: fetching documents or search results and placing them in the model's context before it generates. Retrieval can ground an answer in a real source, but the generated text still has to be checked against that source — attaching a document is not the same as citing it correctly.

Introduced in Unit 3.

plausibility

The property of reading as though it could be right: well-formed, idiomatic, consistent in tone, and shaped like a correct answer. Language-model training and generation strongly reward plausible continuation.

Plausibility is not correctness, and — importantly — it is not evidence of correctness either. A plausible wrong answer can look as polished as a plausible right answer, which is why the module builds a separate means of checking.

Introduced in Unit 3.

hallucination

The established term for output from a generative model that is fluent, well-formed, and false: an invented citation, a non-existent function, a confident date that no source supports.

The word is a metaphor and should not be taken literally. The model is not perceiving anything, not mistaken about anything, and not doing something different from what it does when it is right. It is producing a statistically plausible continuation in both cases; "hallucination" simply names the subset of those continuations that do not correspond to the relevant source, specification, or world. Surface reading may expose some errors, but it cannot establish the claim; that requires appropriate evidence.

Unsupported generation is the more useful working category, because falsity is not the only problem and not the easiest one to detect. Output can be unsupported and accidentally true, more specific than its source, attributed to the wrong source, or simply unverifiable. Truth and evidential support are different questions, and this module is mostly about the second.

Introduced in Unit 3. Used consistently in this sense throughout the module.

independent verification

A check whose evidential basis is separable from the assertion being checked: for example, an authoritative source, an exact calculation, a test oracle built by a different method, or accountable review against explicit criteria.

Re-prompting or self-critique can improve an answer [T9], but the same system may preserve the same assumptions. Improvement is useful; it is not the same claim as independent verification.

Introduced in Unit 3; practised in Unit 8.

convention

A choice about notation or definition that is agreed rather than derived, where a different choice would be equally valid and would give different results. The Bernoulli numbers have two published conventions: the first Bernoulli numbers, with B1 = -1/2, and the second, with B1 = +1/2. Every other value is identical.

Conventions cause a distinctive kind of fault. Two programs can both be correct and defensible while disagreeing at B1, because the specification never said which convention was meant. They agree at every other index, but the unresolved meaning of B1 remains a specification defect. This module fixes the first convention, B1 = -1/2, and states it in the contract.

Introduced in Unit 6. Failure-mode code: b1-sign.

exact rational arithmetic

Arithmetic on fractions represented as an exact numerator and denominator, with no rounding at any point. Python's fractions.Fraction does this: Fraction(1,3) is one third, not an approximation to it.

The lab oracle uses exact rational arithmetic because Bernoulli numbers have large numerators and denominators — B30 = 8615841276005/14322 — and comparing generated output against a reference is only meaningful if both sides are exact. An approximate comparison has to choose a tolerance, and any tolerance loose enough to pass rounding error is loose enough to pass a real bug.

Introduced in Unit 6.

floating point

The binary approximation to real numbers used by default in most programming languages, including Python's float. It is fast, fixed in size, and inexact: most fractions cannot be represented, so results carry small errors that accumulate.

For small Bernoulli indices, floating-point output looks convincing — -0.03333... really is close to -1/30. By B30 the error is unmistakable. Producing floats where exact values were specified is a failure of the specification's arithmetic requirement, not a rounding detail to be tolerated.

Introduced in Unit 6. Failure-mode code: not-fraction.

ground truth

The independently established answer against which a result is judged. In this module, ground truth for the Bernoulli numbers comes from published tables and from the value you derive yourself by hand in Unit 6 — not from the program being tested, and not from a second generated answer that agrees with the first.

The ordering matters: establish ground truth before you look at the output you are judging, or you will find yourself deciding that whatever you got must have been right. This is a discipline rather than a law — a reference sometimes has to be revised once you understand the problem better, and Unit 16 does exactly that. What makes a revision honest is that it leaves a record: what changed, when, and on what evidence.

Some questions have no ground truth of this kind. Unit 15's representation questions have relevant empirical distributions but no table of what an output should have said, and fairness is a normative question rather than a factual one. "No ground truth" there means no single correct answer to compare against, not that nothing can be checked.

Introduced in Unit 6.

off-by-one

An indexing error in which every value is correct but shifted by one position, so that item n returns what belongs at n+1 or n-1. It is the archetypal programming mistake because it survives inspection: the arithmetic is right, the values are all genuine, and nothing looks wrong until you compare index by index against a reference.

Introduced in Unit 7. Failure-mode code: off-by-one.

verification

Establishing that a result is correct by checking it against something independent of the process that produced it. Reading the output again, asking the model whether it is sure, or noting that it looks reasonable are none of them verification, because none of them is independent.

In this module verification takes three concrete forms, all equally valid: comparing against a table you filled in by hand, comparing against a published source, and comparing against a test oracle by running code.

Named in Unit 0 as part of the learning contract; practised throughout and made central in Unit 8.

test oracle

A trusted source of correct answers, used to judge a piece of code under test. course/lab/reference_bernoulli.py is this module's oracle.

An oracle is only worth as much as its own validation, so this one is checked in two directions: against 22 values transcribed from published tables, which it did not compute, and against properties that must hold regardless of how it computes anything.

Introduced in Unit 8.

differential testing

Running two implementations of the same specification on the same inputs and comparing the results, on the principle that two independent routes to the same answer are much stronger evidence than one route agreeing with itself. course/lab/check.py does this: your routine against the oracle, index by index.

The independence is what carries the weight. examples/correct_example.py deliberately uses a different algorithm from the oracle — the Akiyama–Tanigawa transform rather than the binomial recurrence — so that agreement is informative rather than circular.

Introduced in Unit 8.

property-based check

A check that tests a property which must hold for every valid input, rather than comparing specific input-output pairs. The module's example: every odd Bernoulli number above B1 is exactly zero, so B_(2k+1) = 0 for all k >= 1.

A property check catches faults a hand-picked table of examples misses, because the property is stated for the whole domain rather than for the rows someone happened to write down. It still only tests the inputs it runs: this module checks the property over its stated range, so it is an invariant check over that range, not a proof for every k. Elsewhere "property-based testing" usually means a tool that generates many inputs to attack the property; the difference is how the inputs are chosen, not what is being asserted.

Introduced in Unit 8. Failure-mode code: odd-terms.

first divergence

The lowest index at which a routine under test and the oracle disagree. The checker reports this specifically because it gives a reproducible boundary: every lower tested input matched. It is often a useful place to start debugging, but it does not prove that later failures share the same cause.

Introduced in Unit 8.

overreliance

Depending on a tool beyond the point where you can tell whether its output is right. It is not the same as using AI heavily; it is using AI in place of the understanding that would let you check it.

The practical test is whether you could detect an error if there were one. If a generated answer would look identical to you whether it was correct or not, you are relying on it rather than using it. This is why the module has you derive B4 by hand before any code is generated: the understanding has to come first, or there is nothing to check against.

Introduced in Unit 10.

established, contested, unsourced

The module's three buckets for the status of a claim. Established: you can name where it comes from, and the source is one that you, or someone accountable to you, has actually looked at. Contested: sources disagree, or a single account has been copied forward through many retellings without an independent check behind it. Unsourced: you cannot currently say where the claim came from — which is not the same as false.

A claim's bucket is part of the claim, and a claim moves buckets when a check is actually done, not when it is repeated more.

Introduced in Unit 16, §6 of its reading; the same status labels are used throughout course/references.md.

disclosure

A short, factual statement of how AI was used in producing a piece of work: what was generated, at what stage, and what was done with it afterwards. Disclosure answers the question how was this made.

It is a statement about provenance. It is not a claim about quality, and it does not remove the user's responsibility for the output. Other people and organisations may also have duties in a wider system.

Named in Unit 0; templated and practised in Unit 17. See course/learner-evidence-template.md.

defence

A short statement of why the work is correct, citing the checks that were run and the reasoning that was followed. Defence answers the question why is this right.

A defence rests on evidence: a value derived by hand, a table compared, a checker run and its output. It never rests on the authority of the tool that produced the work. "The model was confident" and "the code looked right" are not defences.

Disclosure and defence are separate statements answering separate questions, and the module asks for both. The gap between them is the gap between "I generated this" and "I understand and can defend this".

Introduced in Unit 17.

provenance

Where a thing came from: who or what produced it, by what process, at what stage, and what happened to it afterwards. A disclosure is a statement of provenance. Provenance is worth recording honestly, and it is not evidence of correctness — knowing how a piece of work was made tells you what to check, not whether it is right.

Introduced in Unit 17.

warrant

What entitles you to assert a claim: evidence you can show — a value derived by hand, a comparison recorded, a checker's output — together with the specification or assumption the evidence was gathered against. Warrant is about what you can point at, not about who produced the text. A defence carries warrant; a disclosure carries provenance; neither substitutes for the other.

Introduced in Unit 17; taken up again in Unit 18, where a warranted claim carries its index — who observed, in what context, with what record.

first-order and second-order reasoning

In Unit 18, first-order reasoning describes an observed system from the outside; second-order reasoning includes the observer in the system being described, so that claims carry a three-part index: who observed, in what context, with what record.

This use of "second-order" comes from the second-order cybernetics tradition [C1], developed from the Macy Conferences onwards. It is not the distinction between first-order and second-order logic, and it is not presented as settled terminology across every field.

Introduced in Unit 18.

observer

In Unit 18's sense, whoever or whatever is doing the describing or the checking. The observer question is: when you check something, are you outside the system you are checking, or part of it? Including the observer makes a claim more checkable, not less, because an indexed claim — who observed, in what context, with what record — says what would have to be compared with what. Maturana's compact version: anything said is said by an observer [C3, p. 8].

Introduced in Unit 18.

Macy Conferences

Ten meetings on cybernetics held between 1946 and 1953, funded by the Josiah Macy Jr. Foundation and chaired by Warren McCulloch [C4], with deliberately mixed participants — among them Wiener, von Neumann, Shannon, Bateson, Mead and von Foerster. The reflexive question behind second-order cybernetics surfaced there and was developed afterwards. A recognised research tradition with a literature — and a tradition rather than a theorem.

Introduced in Unit 18.

understanding

This module uses a working structural account, not a settled definition: your internal model of a target — another agent, a process, a piece of code, or a theory — supports understanding when its parts, relationships, and predictions continue to correspond reliably with the target under relevant checks. The verification lab gathers evidence for that correspondence through input–output agreement, property checks, and the first divergence. It does not measure understanding directly or completely.

Understanding is separate from correctness. It is a structural property — how well your model maps the target — not a claim that the target is right. You can understand a flawed framework perfectly: the module's authors mapped one faulty reading of Note G's table and predicted exactly what it computes, while misnaming which reading they had mapped. The capacity to map (understanding) and the empirical accuracy of the map (correctness) are two different things, and a defence needs both — plus a statement of what the map is a map of.

For the formal statement — understanding as a structure-preserving map f : M1 → M2 whose validity is measured by consistency, and its decoupling from truth — see course/second-order.md. Causal accounts emphasise explaining interventions; counterfactual accounts emphasise predicting what would change under different conditions. They are alternatives and possible additions; this module uses the structural account without claiming to settle the question.

Introduced in Unit 17; formalised through the second-order material in Unit 18.

formative and summative

Formative work exists to tell you — and possibly your instructor — what you have understood while there is still time to act on it. It is not marked, or is marked only for feedback.

Summative work is assessed and contributes to a result.

The self-check in course/lab/ and the browser route are formative and unmarked unless your instructor states otherwise. The distinction matters for how you should use them: on formative work, discovering that you were wrong is the useful outcome, and a checkpoint you failed twice before passing has taught you more than one you passed first time.

Introduced in Unit 0.

task class

Where the reference for a claim lives, which is what decides how it can be checked. Unit 1 sorts work into retrieval (the reference is a source you can open), transformation (the reference is your own input), computation (the reference is a rule you can apply), and judgement (there is no single correct answer).

The class gives the starting estimate of what checking will cost. Consequence, access to sources, the size of the task, your own expertise and how current the evidence has to be all move it from there.

Introduced in Unit 1.

context window

Everything a system has in front of it when it produces the next piece of output: your prompt, instructions the product supplied, earlier turns, attachments, and anything a tool fetched. It is bounded, and what happens when the bound is reached — truncation, summarising, selective retrieval — varies by product.

Introduced in Unit 5.

attachment

A file supplied to a system alongside a prompt. For privacy purposes the whole file has left your device; whether all of it reaches the model is a separate question, since products may index, chunk or extract from it.

Introduced in Unit 5; the privacy consequences are Unit 13.

tool call

A system invoking something outside itself — a search, a calculation, a file operation, an API — and receiving a result. The selection of tool and arguments is generated; the execution runs under the tool's own semantics and against the state of the world at that moment, so it can fail, half-succeed, or succeed while doing the wrong thing.

Introduced in Unit 4; central to Unit 11.

agent

A system that acts rather than answers: it calls tools in a loop, and some of those calls change something outside the conversation. What changes for you is that the world may move before you have looked at anything.

Introduced in Unit 11.

trace and log

A log is the record a system wrote as it ran: calls, arguments, results. A trace as displayed in a product may be that log, a condensed rendering of it, or a generated account of what happened — and the three look alike on screen. A log is evidence; a generated account of a run is generated text.

Introduced in Unit 11.

reversibility and recovery

Four different things that "undo" can mean, costing different amounts. Undo reverses the action. A compensating action leaves it standing and offsets it — a refund, a correction. Recovery rebuilds state from something kept for the purpose, and only exists if someone arranged it beforehand. Containment limits what follows when none of the others is available.

Introduced in Unit 11; generalised in Unit 12.

escalation

Handing a decision to someone qualified or authorised to make it. It is a routine professional move rather than an admission, and it is triggered by the situation — consequence, irreversibility, conflicting evidence, whose decision it is — not by the subject matter.

Introduced in Unit 12.

competence and authority

Competence is whether you could recognise a wrong answer. Authority is whether the decision is yours to make. They are independent: you can understand a clause perfectly and still not be the person who may accept it, and either one missing is a reason to escalate.

Introduced in Unit 12.

data minimisation

Sending the least that will do. Usually the practical move is not "do not use the tool" but "change what you send" — generalise, abstract the question, or supply a rewritten fragment rather than the document.

Introduced in Unit 13.

redaction and re-identification

Redaction removes identifying details. Re-identification is recovering a person from what is left, which a combination of ordinary details often permits — a role, a date and a place may be enough. Redaction reduces risk; it does not by itself make material anonymous, nor make its use authorised.

Introduced in Unit 13.

locator

The part of a citation that says where in a source to look: page, section, clause, timestamp. A locator is what makes a citation checkable in bounded time, which is why Unit 9 asks for one and why a fabricated locator is its own failure mode.

Introduced in Unit 9.

source-checking failure modes

The six named ways a generated citation fails, in decision order: fabricated-locator, stale-version, misattributed, over-specified, unsupported-inference, right-source-wrong-claim. No program assigns them — there is no oracle for whether a source supports a claim — so the discipline is the closed vocabulary and the order in which you ask.

Introduced in Unit 9; defined in docs/AUTHORING.md §3.2.

representation

Whose lives, names, places and practices appear in generated material, and in what proportions. Output tends to reflect the distribution of its training data as later training and system design allow. Two failures matter: a visible skew, and an absence — and absences do not announce themselves.

Introduced in Unit 15.

accessibility

Whether material can be used by people with different vision, hearing, motor control and attention. Much of it is checkable today with a tool: alt text that conveys the image's purpose, decorative images marked as such, headings that follow the structure, sufficient contrast, meaningful link text, keyboard access with visible focus. Automated checks are necessary and not sufficient, and a green result is not a conformance claim.

Introduced in Unit 15.