Unit 9: Alles Lüge! First- and second-order reasoning

The module's closing unit, and the last required one. No new practical work: it names the question the earlier units kept running into, says where that question comes from, and closes off the three overclaims easiest to reach from it. Units 0 to 8 make complete sense without it.

Reading time: about 12 minutes. Activity: about 12 minutes. Course items: 1, 2, and 4. Checkpoint: cp09-sort-the-claims. Before this unit: Unit 8 (disclosure, defence, and the evidence pack).

No prior study of logic or cybernetics is assumed. Every specialist term needed for the activity is defined below; the optional further reading carries the advanced detail.

What this unit does, and what it does not do

This unit covers established, published material: a distinction from cybernetics, where it came from, and two classical theorems about self-reference. Its job is to raise the issue and say what "second order" means. It proves nothing about language models and teaches no unpublished research. That restriction is deliberate: the module has spent three hours training you to ask what supports a claim, and a closing unit that quietly imported an unsettled framework would fail its own test in front of you.

Bracket labels — [C1], [C4], and so on — resolve to course/references.md, and course/second-order.md gives the fuller treatment this page summarises.

1. What you have already done

Five things, all done with your hands before you met any of the language here.

• B4 derived from the recurrence on paper, before you saw any generated code.
• The checker reporting b1-sign rather than "wrong".
• Two valid Bernoulli conventions disagreeing at B1 and nowhere else, because neither program had said which one it used.
• Ada's B7 and the modern B7: one label, two different terms.
• A disclosure and a defence, written separately.

Section 3 says how each of those reads in this unit's vocabulary. Underneath all five is one question: where is the observer standing? When you check something, are you outside the system you are checking, or part of it? That question did not begin with artificial intelligence, and it has a substantial published literature going back to the 1940s. The rest of this page is what that literature calls it.

In the plain language used here:

• the observer is the person or system making a claim or carrying out a check;
• the context is the convention, specification, setting, and limits under which the claim is made; and
• the record is the retained evidence of what was observed or checked.

These three details make a claim easier to evaluate. They say who did what, under which conditions, and where the evidence can be inspected.

2. First order and second order

The distinction comes from cybernetics and is usually put like this.

• First-order cybernetics — the cybernetics of observed systems. The observer stands outside, describing a system without being part of it.
• Second-order cybernetics — the cybernetics of observing systems. The observer is included in the system being described.

Heinz von Foerster formulated the distinction in those terms [C1]. Margaret Mead's 1968 address "Cybernetics of Cybernetics" is commonly cited as the point where the reflexive turn was named [C2].

The consequence is small to state and large in effect. An observer inside the system is not taking a free look from nowhere: observing takes time, costs something, and can change what is observed. Claims stop being simply true or false and start carrying an index — who observed, in what context, with what record. Maturana's compact version: anything said is said by an observer [C3, p. 8].

A note on precision, since this unit is about exactly that. This line is very often quoted as everything said is said by an observer. It is not: page 8 of [C3] reads anything. The everything form is the title of a later Maturana essay (the 1987 piece in Thompson's Gaia: A Way of Knowing; see the note under [C3] in course/references.md), and it has propagated by repetition into places that cite the 1980 book for the wording it does not contain.

Two notes on the vocabulary, because both catch people out.

• This is not the first-order/second-order distinction in logic. The same words name different contrasts in different fields; this page means the cybernetic one.
• Including the observer makes a claim more checkable, not less: an indexed claim says what would have to be compared with what. Naming the Bernoulli convention turned an unresolvable disagreement into a resolved one.

Where it comes from

The Macy Conferences on Cybernetics were ten meetings held between 1946 and 1953, funded by the Josiah Macy Jr. Foundation and chaired by Warren McCulloch [C4]. Their mixed disciplines and the later development of the reflexive question are described in the optional further-context section.

Second-order cybernetics is a recognised research tradition with a literature, not a fringe or recent position. It is also a tradition rather than a theorem — a category, not a criticism. A programme can be decades old, published and influential while remaining a way of framing problems rather than something anyone proved. Telling those apart is the exercise at the end of this unit.

Keep two kinds of claim separate. Second-order cybernetics supplies an interpretive lens: a published way to frame observer-dependent problems. The logic results in section 4 are proved theorems within stated formal conditions. The lens does not follow from the theorems, and the theorems do not prove the module's interpretation.

3. The mapping, and what it is not

Adapted from course/second-order.md §3. The left column is what you did; the right is a reading of it.

A reading, not a proof. None of it is a deduction from the theorems in section 4. Reject the whole second-order framing and you still have a working oracle, a checker, and a correct routine.

One working account of understanding, not a settled definition. On the structural account used in this module, your internal model supports understanding when its parts, relationships, and predictions continue to correspond reliably with the target under relevant checks. The lab gathers evidence for that correspondence through predicted values, property checks, and the first place two records diverge.

That capacity is separate from the target's empirical correctness. You can model a wrong table accurately, as the module's authors did with the as-printed Note G table. In that case you understand what the table says without making the table right. A defence therefore needs evidence both that your model corresponds to its target and that the target is appropriate and correct for the claim you are making.

Other accounts emphasise causes and counterfactuals: understanding includes being able to say what would happen after an intervention, or what would change under a different condition. These are alternatives and useful additions, not positions this short module settles. course/second-order.md gives the optional formal structural statement (f : M1 → M2) and its limits. The practical rule remains: pause when you can no longer understand and defend what is being produced. In operational terms, pause if you cannot predict relevant behaviour, explain the transformation, identify an independent check, or interpret a failed check. Confidence is not a fifth form of evidence.

4. The classical results on self-reference

Three plain definitions come first:

• A formal system uses explicit symbols, starting assumptions, and rules for deriving one statement from others.
• A theorem is a statement established by proof under stated rules and assumptions.
• A truth predicate is an expression inside a language that would classify sentences of that language as true or not true.

Two established theorems are the rigorous core of "a system cannot fully account for itself from inside", and their limits matter as much as their content.

• Gödel's incompleteness theorems (1931): any consistent formal system strong enough for arithmetic contains true statements it cannot prove [C6].
• Tarski's undefinability theorem (1936): such a system cannot define a truth predicate for its own sentences [C7].

What they do not say

This subsection matters more than the two lines above it, so read it slowly.

Both are results about formal systems of a specific kind: a formal language with a defined satisfaction relation and enough expressive power to talk about its own sentences. They do not straightforwardly transfer to language models, to institutions, or to people, and this unit does not claim they do. Their role is to establish that limits on self-reference are real and provable somewhere — much weaker, and much safer, than asserting a limit on a particular AI system.

Anyone who tells you that Gödel proves something about machine intelligence is making a leap. The conclusion may still hold; it has not been shown, because the theorem is about a different kind of object, and the step across is an argument someone must make and defend. Noticing that leap — in a paragraph that is fluent, confident, and cites a real theorem — is the skill this module teaches.

5. What this does not license

Three overclaims are easy to reach from here. This unit closes each one off.

"Nothing can be known" — overclaim

The move is from a guarantee to a method: state your context, keep records, compare independent ones. The module is its own counterexample. You finished Unit 5 with evidence that your routine agreed with the oracle over the tested indices and satisfied the checked properties under a stated convention. That scoped conclusion is stronger than felt certainty because it states its limits. The module itself was made by that same method, not exempted from it: generated with AI assistance and then held to external checks before it reached you, with its AI-use disclosure recorded once, at module level, in the repository, as the honest record of how — which is the difference between trusting it and taking it on faith.

"Und doch ich hör' nur Lügen" — and yet I hear only lies — (Lacrimosa, "Alles Lüge") [Q1]. That is the position this section refuses. Total distrust is not scepticism but a way of stopping checking: someone who doubts everything cannot decide where to spend the effort.

"Therefore AI cannot be trusted at all" — overclaim

The observer question applies with equal force to human reasoners, to the oracle, to peer review, and to the published Note G table that was wrong for a long time. Nothing here establishes how often a language model is right or wrong. The response is architectural, not attitudinal: put the check outside the thing being checked, index the claim, say what kind of support you have.

"Gödel and Tarski prove that AI cannot verify itself" — overclaim

They do not; see section 4. This is the most interesting of the three, because both halves are true and the join is not: real theorems, real systems, and an unstated argument between them. That shape reads extremely well, and it is exactly what Unit 4 trained you to catch.

6. Further context (optional)

Nothing in this section is required for the activity or checkpoint.

People and disciplines around the Macy Conferences

Participants came from mathematics, engineering, neurophysiology, anthropology, and psychiatry. They included Norbert Wiener, John von Neumann, Claude Shannon, Gregory Bateson, Margaret Mead, and Heinz von Foerster [C4]. The reflexive question was developed afterwards by von Foerster and the Biological Computer Laboratory [C1], Bateson [C5], and Maturana and Varela [C3].

For the complete bibliography, use course/references.md: [C1] to [C5] cover second-order cybernetics and its history; [C6] and [C7] cover Gödel and Tarski. The fuller conceptual treatment is in course/second-order.md.

The register, and the rule that governs it

This is the only unit that uses the module's gothic register at full strength, including two short song quotations. It is a deliberate choice with a justification, not decoration: Ada Lovelace was Byron's daughter [G1], and the 1816 Villa Diodati gathering is where the ghost-story challenge associated with Frankenstein [G3] and Polidori's The Vampyre [G4] is conventionally dated [G2]. The connection is thematic; no claim is made that this literature influenced Lovelace's work.

Notice what those citations carry. [G2] is honest that the nearest thing to a primary account is Mary Shelley's own introduction to the 1831 edition, and it still records the date of the ghost-story challenge as contested — a range, not a night. So: connection supported, summer conventionally dated rather than documented to the day, influence claim not made.

"It's not the red of which we bleed" (Einstürzende Neubauten, "Sabrina") [Q2] — a note about warrant. Nothing here descends from above. The red accent in these materials always means divergence and never travels without a word beside it; warrant is assembled the same way, locally, from records you can point at, by someone who can be named. Both quotations are short, attributed and used for commentary; their entries and the rights position governing their survival to release are in course/references.md [Q1], [Q2].

The one rule. The register may carry the argument and may never substitute for it. Every claim on an atmospheric slide still carries its citation or its uncertainty label. If a sentence in this unit sounds good and cites nothing, it is a defect, and you are entitled to say so.

7. What you can now do

• Say what the first-order/second-order distinction is, and where it came from.
• Name three things you did in this module that the second-order reading illuminates, while saying plainly that illumination is not proof.
• State Gödel's and Tarski's results within their actual scope, and say why neither settles anything about a language model on its own.
• Tell a proved theorem from an established research programme from an overclaim, when all three appear on the same well-written page.

Checkpoint

Work through activity.md first: the checkpoint asks you to sort claims made by this unit, and it is a different exercise once you have sorted some yourself. It is the last thing the module asks of you. From course/lab/:

python3 selfcheck.py run cp09-sort-the-claims     # or use the browser self-check

Both routes ask the same question and give the same diagnostics.

Where to look next

• course/second-order.md — the fuller treatment this unit summarises.
• course/references.md — full entries for [C1] to [C7], the gothic-context entries, the two song quotations, and the source policy.
• course/course-map.md — where this module stops and the wider course continues.