# Learner evidence template By the end of this course you should hold a short record of one piece of work you generated, checked, corrected, and can defend. That record is your evidence pack. It is short. It is not an essay, and it is not a programming assignment. Its purpose is to make visible the thing the course is actually about: the distance between "I generated this" and "I understand and can defend this". **The pack is yours and stays yours.** It is a private formative self-check. You are not asked to submit it, it is not marked, and nobody collects it. Export it, print it, or show it to someone only if you decide to. ## Three routes, one pack There are three ways to fill this in, and they produce evidence of equal standing. - **Table-based verification.** You compare generated values against a reference table — one you derived by hand and one published in the course — and record what matched and what did not. No code is run at any point. - **Automated verification.** You run `course/lab/check.py` against a Python file and record what the checker reported. - **Source-based verification.** You take a referenced summary, retrieve what the cited sources actually say, compare claim by claim, and record the first claim the source does not support together with its named failure mode. No code is run at any point, and the artefact under test is prose rather than a routine. Unit 9 works this through. All three routes verify a claim against a reference you established independently, which is what verification means. They differ in what the artefact is and in who makes the final judgement, not in standing. The first two are worked through below in full; the third produces a comparison table in place of checker output, and everything this template asks for applies to it unchanged. Pick the one that suits how you work and what you are checking. If you would rather not install or run Python but want a runnable checkpoint record, the browser self-check on your progress page produces one; the Python route produces the same record with `python3 selfcheck.py progress --evidence`. --- ## What goes in the pack, and where it comes from Generated from `course/evidence-pack.yml`, which is the canonical list. Units fill the pack in as you go; Unit 17 assembles it. If a unit's own wording and this table ever disagree, this table is the one to believe. | Pack item | From | Routes | What to keep | |---|---|---|---| | before §1 Verification commitment | Unit 0 | all three | The one sentence you wrote naming an artefact, a check, and what would count as failure. Unit 18 asks you to look back at it. | | before §1 Task and reference classification | Unit 1 | all three | Your table of a week's tasks sorted by where the reference lives. The first thing in the pack about your own work rather than about the case study. | | §1 Original prompt | Unit 7 | all three | The prompt exactly as you sent it, including anything you got wrong or left out. On the source route, the prompt that produced the referenced summary. | | §2 Generated output | Unit 7 | all three | The output as it arrived, untidied, including any claims about history or sources. | | §3 Verification result | Unit 8 | table, code | The reference you established independently, the method you used, and what the comparison showed. | | §3 Verification result | Unit 9 | source | What the cited sources actually say, retrieved by you, and the claim-by-claim comparison against the summary. | | §4 Discrepancy notes | Unit 8 | table, code | The first divergence, and each discrepancy with its failure-mode code. | | §4 Discrepancy notes | Unit 9 | source | The first claim the source does not support, and each fault with its source-checking code under the stated decision order. | | §5 Revised prompt | Unit 10 | all three | The clauses you added, each traceable to a specific discrepancy rather than to a general wish for a better answer. | | §6 Final result | Unit 10 | all three | The re-checked result, verified by the same method as the first, with the result recorded. | | §7 AI-use disclosure | Unit 17 | all three | How the work was made, in your own words. Provenance only: what the tool did, at which stage, and what you changed. | | §8 Correctness defence | Unit 17 | all three | Two sentences citing checks and their scope, and naming what the check did not establish. Not the tool's confidence, not its name. | | §9 One artefact from your own work | Unit 17 | all three | A piece of AI-assisted work from your own subject, run through the same method. This is the item that transfers. | Some units add to an item rather than creating one. These belong in the pack and attach where they are named: | From | Attaches to | What to keep | |---|---|---| | Unit 11 | §4 | Where in an agent trace you would put a check, and which reading of the trace you were relying on. | | Unit 12 | §8 | Any escalation decision and what you handed over. The pack is what you bring to the qualified person. | | Unit 13 | §7 | What you did not send, and why. A redaction decision is part of how the work was made. | | Unit 14 | §7 | Any rights or attribution decision, which is separate from disclosure and belongs beside it. | | Unit 16 | §4 | Any historical or source claim you checked, with its status: established, contested or unsourced. | | Unit 18 | §8 | Your defence split into record-supported result, reasoned judgement and any overclaim, and the reflection on your Unit 0 commitment. | ## The template Copy everything between the rules into a new file and fill it in. --- ### Evidence pack **Name:** **Date:** **Context:** academic (course or module) / professional (role and organisation) / self-study **Route used:** table-based verification / automated verification / source-based verification **Task:** (one line — what you asked for) #### 1. Original prompt Paste the prompt exactly as you sent it, including anything you got wrong or left out. Do not tidy it up. The gaps in this prompt are the evidence. ```text ``` #### 2. Generated output Paste what came back, or the relevant part of it. If it was code, include it as code. If it was a table of values, include the table. If it carried claims about history or sources, include those too — they are part of the output and part of what needs checking. ```text ``` #### 3. Verification result State what you checked it against, by what method, and what happened. Name your reference. "It looked right" is not a result; "B1 disagreed with the value I derived in Unit 6" is. - **Checked against:** - **Method:** - **Result:** #### 4. Discrepancy notes For each disagreement: what the output said, what the reference said, and — as far as you can tell — what kind of error it was. Use the failure-mode codes the checker reports where one applies (`b1-sign`, `odd-terms`, `ada-indexing`, `off-by-one`, `not-fraction`, `raises`, `import-error`, `no-function`); most are discussed in `course/misconceptions.md`. If nothing disagreed, say so, and say what you checked that *could* have disagreed. A check that could not have failed has not told you anything. | index or claim | output said | reference said | code / kind of error | |---|---|---|---| | | | | | #### 5. Revised prompt Paste your second prompt. Then, in one or two lines per clause, say which clause you added and which discrepancy above it was meant to remove. Every clause should be traceable to something that actually went wrong. ```text ``` | clause added | discrepancy it addresses | |---|---| | | | #### 6. Final result The corrected output, and the verification you ran on it. Same standard as section 3: name the reference and the method. ```text ``` - **Checked against:** - **Method:** - **Result:** #### 7. AI-use disclosure Two or three sentences. What you used AI for, at what stage, and what you did with the output afterwards. Factual and specific. This is a statement about how the work was made. > #### 8. Correctness defence Exactly two sentences. Why the final result is correct, citing the checks you ran or the reasoning you followed. This is a statement about why the work is right, and it must not rest on the authority of the tool. > --- #### 9. One artefact from your own work Everything above concerns a case where "correct" is decidable. This item is the one that transfers: a piece of AI-assisted work from your own subject, role or study, planned through the same method. It becomes a completed verification when you run and record the check. Five short lines, from Unit 17, Task 4: - **The artefact:** (what it is, and what it would be used for) - **Kind of claim:** exact and checkable / source-dependent / judgement / low-risk draft — more than one may apply, and privacy or consequence overrides "low risk" - **What could fail, and its symptom:** - **The check, its reference, and how long it takes:** (if no independent reference exists, say so — that is a finding) - **What you could then honestly claim, and what stays unchecked:** Nobody collects this. If you have a tutor, colleague or study partner, this is the item worth discussing with them: it shows what you did, not what you produced. --- ## Worked illustration A — table-based verification *A completed pack, no code run at any point.* **Route used:** table-based verification **Task:** Get a table of the Bernoulli numbers `B0` to `B8` to use in a report. **1. Original prompt** ```text Give me a table of the first few Bernoulli numbers. ``` **2. Generated output** ```text B0 = 1 B1 = 1/6 B2 = 0 B3 = -1/30 B4 = 0 B5 = 1/42 B6 = 0 B7 = -1/30 These are the values Ada Lovelace computed in Note G of her 1843 notes, where she calculated B7 using the Analytical Engine. ``` **3. Verification result** - **Checked against:** the value of `B4` I derived by hand from the recurrence in Unit 6, and the course's published table in `course/lab/README.md`. - **Method:** wrote both sequences side by side and compared index by index. - **Result:** agrees at `B0` and disagrees from `B1` onwards. My hand derivation gives `B4 = -1/30`; the output files `0` there and puts `-1/30` at `B3`. **4. Discrepancy notes** | index or claim | output said | reference said | code / kind of error | |---|---|---|---| | B0 | 1 | 1 | agrees — and this is the one index that matters, see below | | B1 | 1/6 | -1/2 | `ada-indexing` — the modern B2, filed as B1 | | B3 | -1/30 | 0 | `odd-terms` — odd terms above B1 must be exactly zero | | B4 | 0 | -1/30 | the value that belongs here has been filed at B3 | | B1, B3, B5, B7 as a block | the non-zero values in order | modern B2, B4, B6, B8 | `ada-indexing` — Note G's labels | | "she calculated B7" | Note G computes B7 | Note G computes *Ada's* B7, the modern B8 | index confusion in prose, unsourced | | "using the Analytical Engine" | the engine ran the computation | the engine was never completed (Unit 2) | unsourced historical claim | The whole table is Lovelace's numbering: every value in it is a genuine Bernoulli number, filed under Note G's labels rather than modern ones — her `B1, B3, B5, B7` are the modern `B2, B4, B6, B8`, and she has no `B2` at all, so the even indices here are empty. Against the modern convention that is inverted. **And the agreement at `B0` is the whole of my evidence for saying so.** A plain off-by-one shift would produce exactly this output at every other index, because her `B_n` at odd `n` is the modern `B_(n+1)` and both give zero at the even ones. What separates them is that a shift would put the modern `B1 = -1/2` at `B0`, and this output leaves `1` there. One index is thin evidence, and I am recording it as the reason rather than as a certainty: to settle it I would read the source. The historical sentence is a separate claim which the correct-looking arithmetic gave me no reason to trust, and the exemplar verifier catches it at the first source check: the output says the computation was run "using the Analytical Engine", but the engine was never completed (Unit 2) — see `course/misconceptions.md` §6. **5. Revised prompt** ```text List the Bernoulli numbers B0 to B8 using modern indexing, where every index is numbered including the odd ones that are zero. Use the first Bernoulli convention, B1 = -1/2. Give each value as an exact fraction, not a decimal. Do not include any historical commentary. ``` | clause added | discrepancy it addresses | |---|---| | "modern indexing ... including the odd ones that are zero" | `ada-indexing`, `odd-terms` | | "first Bernoulli convention, B1 = -1/2" | pre-empts `b1-sign`: the first output had no modern B1 to judge | | "exact fraction, not a decimal" | pre-empts `not-fraction` | | "no historical commentary" | the unsourced Note G claim | **6. Final result** ```text B0 = 1 B5 = 0 B1 = -1/2 B6 = 1/42 B2 = 1/6 B7 = 0 B3 = 0 B8 = -1/30 B4 = -1/30 ``` - **Checked against:** my hand-derived `B4`, and the course's published table in `course/lab/README.md`, which itself agrees with 22 values transcribed from published tables. - **Method:** index-by-index comparison of all nine values, twice, second time reading the reference first. - **Result:** all nine agree. The odd terms `B3`, `B5`, `B7` are now zero, which is the property I had specifically failed on before. **7. AI-use disclosure** > I used an AI assistant to produce a table of Bernoulli numbers `B0` to `B8`. > The first output used Lovelace's 19th-century indexing and so had no modern > `B1` at all, which I found by comparing it against a value I had derived by hand > and against the course's published table. I re-prompted with the convention and > indexing stated explicitly and verified the second table the same way. **8. Correctness defence** > Every value in the final table matches the course's published reference table > index by index, and `B4 = -1/30` matches the value I derived independently from > the recurrence before generating anything. The convention is stated — first > Bernoulli numbers, `B1 = -1/2` — so the comparison is between two tables that > are answering the same question. --- ## Worked illustration B — automated verification *The same task, checked by running the code.* **Route used:** automated verification **Task:** Get a Python function `bernoulli(n)` for use in the Unit 8 exercise. **1. Original prompt** ```text Write me a Python function to compute Bernoulli numbers. ``` **2. Generated output** ```python from fractions import Fraction from math import comb def bernoulli(n: int) -> Fraction: values = [Fraction(0)] * (n + 1) for m in range(n + 1): values[m] = Fraction(1) for k in range(m): values[m] -= Fraction(comb(m, k) * values[k], m - k + 1) return values[n] ``` I read the file before running it: it imports only `fractions` and `math`, and does nothing to the filesystem, the network, or the shell. **3. Verification result** - **Checked against:** `course/lab/reference_bernoulli.py`, the course's test oracle. - **Method:** pasted the function into `exercises/ex04_bernoulli.py` and ran `python3 check.py exercises/ex04_bernoulli.py`, comparing indices 0 to 30. - **Result:** FAIL, two findings. ```text FAIL - 2 problem(s) found. [divergence] First divergence at n=1: expected -1/2, got 1/2. [b1-sign] B1 has the wrong sign for this course's convention. ``` **4. Discrepancy notes** | index or claim | output said | reference said | code / kind of error | |---|---|---|---| | B1 | 1/2 | -1/2 | `b1-sign` | | B0, B2..B30 | — | — | agree exactly | One index out of 31. The routine is correct under the second Bernoulli convention; the specification I gave never said which convention I wanted, so this is a specification failure rather than an arithmetic one. Worth noting that the failure was found at `n=1`, the second value tested — a spot check starting at `B2` would have missed it entirely. **5. Revised prompt** ```text Write a Python function with this interface: bernoulli(n: int) -> fractions.Fraction Defined for every n >= 0; first-Bernoulli convention B1 = -1/2; exact Fraction returns only, never float; returns Fraction(0) for odd n > 1. ``` | clause added | discrepancy it addresses | |---|---| | "first-Bernoulli convention B1 = -1/2" | `b1-sign` — the one that actually failed | | "exact Fraction ... never float" | pre-empts `not-fraction` | | "defined for every n >= 0" | pre-empts `raises` at the boundaries | | "returns Fraction(0) for odd n > 1" | pre-empts `odd-terms` and `ada-indexing` | The interface contract is the course's fixed specification, the same wording `check.py` enforces. Only the first clause fixed a bug I had; the other three are clauses I now know to include because I read the failure modes in `course/misconceptions.md`. **6. Final result** ```python from fractions import Fraction from math import comb def bernoulli(n: int) -> Fraction: if n < 0: raise ValueError("n must be >= 0") if n > 1 and n % 2 == 1: return Fraction(0) values = [Fraction(0)] * (n + 1) for m in range(n + 1): values[m] = Fraction(1) for k in range(m): values[m] -= Fraction(comb(m, k) * values[k], m - k + 1) if n == 1: return -values[n] return values[n] ``` - **Checked against:** the same oracle. - **Method:** `python3 check.py exercises/ex04_bernoulli.py --verbose`, then `python3 selfcheck.py run cp08-bernoulli`; on the table route, complete `cp08-table-evidence` and retain the compared rows. - **Result:** PASS. ```text PASS - agrees with the oracle for every n from 0 to 30, returns exact Fractions, and odd terms above B1 are zero. (Indices compared: 0 to 30) ``` I also derived `B4 = -1/30` by hand in Unit 6, before any of this, and the function returns that value. **7. AI-use disclosure** > I used an AI assistant to write a Python function computing the Bernoulli > numbers. The first version used the second Bernoulli convention, which the > lab's differential checker identified at `n = 1`. I re-prompted with the > course's interface contract stated in full, read the result, and re-ran the > checker. **8. Correctness defence** > The function agrees with the lab's test oracle for every index from 0 to 30, > returns exact `Fraction` values, and returns zero at every odd index above > `B1`; the oracle itself agrees with 22 values transcribed from published tables > and uses a different algorithm from this implementation. `B4 = -1/30` also > matches the value I derived by hand from the recurrence before generating any > code, so the comparison does not depend on the oracle alone. --- ## Completion criteria Your evidence pack is complete when all of the following are true. They apply equally to all three routes. - [ ] All nine sections are filled in. - [ ] The original prompt is the one you actually sent, not a tidied version. - [ ] The generated output is included as it arrived, including any claims about history or sources. - [ ] The verification section names a **specific reference** and a **specific method**, not an impression. - [ ] At least one discrepancy is recorded, or — if the first attempt genuinely passed — you have stated what you checked that could have failed. - [ ] Each discrepancy is labelled with a failure-mode code where one applies — the computable codes on the Bernoulli routes, the six source-checking codes on the source-based one. - [ ] Every clause added to the revised prompt is traceable to a specific discrepancy. - [ ] The final result has been verified by the same method as the first, and the result is recorded. - [ ] The disclosure says how the work was made, in your own words. - [ ] The defence is two sentences, cites checks or reasoning, and does not cite the tool's confidence, the tool's name, or the fact that it "looked right". - [ ] A sceptical reader could recheck at least one claim from what is attached, without taking your word for anything. - [ ] You can answer, out loud, what your result depends on that a reader might assume differently — on the Bernoulli routes, which convention you used and why it matters; on the source-based route, which edition or version of the source you read. Optionally attach your self-check record — `python3 selfcheck.py progress --evidence`, or the summary on your progress page. It shows which checkpoints you passed. It is not evidence that you can defend them, which is what sections 7 and 8 are for. ## Instructor note *What this pack is for, and why it is not collected.* **It is not collected.** The pack is a private formative self-check. It is not submitted, not marked, not reviewed and not collected, and no part of the course asks a learner to hand it over. If a learner chooses to export it or show it to someone — in a tutorial, in peer discussion, or in a portfolio of their own — that is their decision. **What it is for.** The pack makes the learner's own verification process visible to them. The valuable content is sections 4 and 5 — the discrepancies found and the clauses added — because those are where checking either happened or did not. A pack whose first attempt failed, and whose discrepancies and defence are recorded honestly, says more than a pack with a working function. **Read the process, not the artefact.** If a learner does ask you to look at a pack, respond to whether verification occurred and whether the defence is evidence-based. Whether the final code runs, and how good the code is, are not the point. A learner who generated broken code, found the bug, named the failure mode, fixed the specification and defended the result has met every objective in this course; so has a learner who found that a cited source does not contain the claimed finding, traced it to another identified source, named it `misattributed`, and said what the citation can and cannot support. A learner who generated correct code first time and wrote "the AI got it right" has met none of them, and the pack should be able to show that difference to the learner themselves. **All three routes count equally.** A table-based pack, an automated pack and a source-based pack are evidence of the same competence. Nothing in the course should reward running Python, and no cohort communication should describe the table-based route as a fallback or as being for anyone in particular. Worked illustration A is included in full, as B is, for this reason; the source-based route's equivalent is the worked specimen and answer key in Unit 9. **Why the stakes are zero by design.** The course asks learners to record their mistakes in writing. That only produces an honest record if there is nothing to lose by writing one down, which is why the pack is private and uncollected rather than submitted for feedback or for a mark. Attaching a mark to the artefact would produce packs in which nothing ever went wrong, and the course would have taught the opposite of its subject. **Common weak spots, worth naming in teaching.** A defence that names the tool rather than the checks. A revised prompt with clauses that do not correspond to any recorded discrepancy — usually copied from the Unit 10 material rather than derived. "It looked correct" appearing in section 3. A discrepancy table with no entries and no statement of what was checked. Each maps to an entry in `course/misconceptions.md`, so a discussion can point at a specific unit and checkpoint rather than at a learner. **Cross-references.** The disclosure wording, the two-sentence defence, and the assembly of this pack are Unit 17. The interface contract quoted in illustration B is the course's fixed specification. Rules about permitted AI use in assessed work are set by the learner's own institution or programme; this course does not set them and does not state them.