The words spoken in the Unit 15 video, scene by scene, as its captions show them, with what the animations show marked [On screen: ...] where they play. Each speaker is named where the speaker changes. The Unit 15 reading covers the same material.
The narration is spoken by synthetic voices made for this course, each shown on screen as a drawn robot. No person is speaking.
Unit 15 — Bias, representation and accessibility
Narrator: Unit 9 ran the method on a case with no computable answer, where the reference was what the sources said. This one is harder still. The reference is not a document at all.
The wrong question
The obvious question is: is this output biased? That invites a yes or no answer to something that does not have one. The question this course always asks is better. What would I compare this against, and what would that comparison establish? That has real answers here. Less comfortable than a value in a table, and the same kind of work.
Two halves
Output reflects the statistical patterns of the training material.
[On screen: A label says the shares are illustrative, not measured, over five abstract categories, A to E. On the left, a panel of training data shows five bars: A long, B and C middling, D short, E very short. Category A is marked common in the data. An arrow labelled 'tends to' leads to an output panel on the right, where five bars form in the same pattern, A again common. Between the panels three boxes appear: post-training, standing instructions, filtering. Then B's output bar shrinks, labelled damped, and C's grows, labelled introduced. Text: nothing decided this; it is a distribution showing through. Later, E is marked rare in the data, and its output bar becomes thin, labelled thin in the output, likelier generic or wrong; the data bar is labelled less to check it against. Output A is outlined as a skew you can sometimes see. Then E fades from the output and the other four output rows spread apart to fill the space five took, leaving no gap, while the training-data panel still shows E. Closing text: an absence does not announce itself; nothing in the output marks where E was.]
So what was common in the data tends to be common in the output. It only tends to. Between the data and what you see sit post-training, standing instructions and filtering. Standing instructions are the instructions a product adds, which you do not normally see. Any of these can damp a pattern, which means weaken it, or introduce one. Nothing decided this. Calling it a decision would describe the system as a person making a choice, which this course rules out. It is a distribution showing through. Image generators show it more visibly. In a 2023 study, prompts that named no group at all produced stereotyped pictures: 99% of one widely used generator's "software developer" images showed people represented as white, against 56% of software developers in US statistics, and asking for the opposite did not reliably undo it. Those were 2022 systems: the point is the mechanism, not the figure.
[On screen: A label says the shares are illustrative, not measured, over five abstract categories, A to E. On the left, a panel of training data shows five bars: A long, B and C middling, D short, E very short. Category A is marked common in the data. An arrow labelled 'tends to' leads to an output panel on the right, where five bars form in the same pattern, A again common. Between the panels three boxes appear: post-training, standing instructions, filtering. Then B's output bar shrinks, labelled damped, and C's grows, labelled introduced. Text: nothing decided this; it is a distribution showing through. Later, E is marked rare in the data, and its output bar becomes thin, labelled thin in the output, likelier generic or wrong; the data bar is labelled less to check it against. Output A is outlined as a skew you can sometimes see. Then E fades from the output and the other four output rows spread apart to fill the space five took, leaving no gap, while the training-data panel still shows E. Closing text: an absence does not announce itself; nothing in the output marks where E was.]
And what was rare in the data is thin in the output. That is the less-discussed half and often the more consequential: material about smaller languages, smaller places, less-documented practices is thinner, so output about them is likelier to be generic or wrong. And there is less to check it against. Which is why "bias" is a poor word for it. One is a skew you can sometimes see in a single output. The other is an absence. And absences do not announce themselves. You will meet both in summaries, in generated examples, and most consequentially in anything that describes or ranks people. A comparison can catch the skew, and the next scenes show how. The absence needs a different kind of check, and it comes after them.
Vary one thing
Questioner: How would I actually check an output for bias?
Narrator: The closest thing to a controlled comparison available, and the one to reach for. Produce the same output with a single attribute changed. A name, a pronoun, a place, a language. Software testers have a name for this: metamorphic testing. Say how the two outputs should relate when one input changes. Then try many cases, looking for one where that fails.
[On screen: A tally with two features down the side and three runs of prompt A and prompt B across. After the first pair both features look like findings: each appears on one side and not the other. As the second and third pairs fill in, the different occupation holds on every run of prompt A and never on B, while the family mention appears once on each side and then not again. The first is marked as holding on every run; the second is marked as the system varying.]
Here is one pair. The tally tracks two features of the output: whether it describes a different occupation, and whether family is mentioned. After one pair, each appears on one side only, so each looks like a finding. But that is one observation with two explanations. Two fresh conversations differ in two ways: the attribute you changed, and the ordinary run-to-run variation. So repeat the pair several times, alternating which goes first, and look at whether a difference persists. Use separate conversations, not the same one: as Unit 5 showed, the same conversation is not a second trial.
What it establishes, and what it does not
Questioner: And after three pairs, what does the comparison show?
[On screen: A tally with two features down the side and three runs of prompt A and prompt B across. After the first pair both features look like findings: each appears on one side and not the other. As the second and third pairs fill in, the different occupation holds on every run of prompt A and never on B, while the family mention appears once on each side and then not again. The first is marked as holding on every run; the second is marked as the system varying.]
Checker: Three pairs, alternating order: one feature held in every run on one side and none on the other; the second appeared once on each side and then did not.
Narrator: That establishes a pattern in these outputs, on this system, on this date. What it does not establish: a rate, a harm, the mechanism, or that no difference exists anywhere else. Six runs are six runs, and with so few even that pattern could be chance; more runs would test it. The opposite mistake, saying there is no problem, is the easier one to make without noticing. "The difference was small, so there is no problem" is a general claim from one comparison, exactly as much as the accusing version is.
A match is not a licence
There is a second comparison to make. Where a published statistic exists, you can compare the output's spread against it. That gives you something specific and citable.
Questioner: So a match with real figures means no problem?
Narrator: No. A match to the real distribution is not permission, because the real distribution may be exactly the thing you were trying not to reproduce.
Who is missing
Both checks so far compared the output with something: the same prompt with one attribute changed, or a published figure. The last check is for the other half, the absence, and it starts differently. It starts as a prompt to yourself. Who is not in this output, and would I have noticed if I had not been asked? An absence is easy to miss when your reference omits the same groups. So where you can, also compare with a list of relevant groups assembled independently of the output.
Accessibility is the checkable part
Some of this unit is genuinely hard to establish. This part is easier. Start with alt text. It is a short written description attached to an image. A screen reader reads it aloud for someone who cannot see the image, and a browser shows it when the image does not load. It matters most to blind and low-vision readers, and it also serves search engines and anyone on a slow connection. A good one describes function: what the image is for in that place, and what the reader is meant to take from it, not just what it looks like. But generated alt text tends to describe contents, because the image is in the request and your reason for including it is not. That is not a criticism of the generated version. It is an accurate description answering the only question it could answer. A decorative image takes empty alt text. A chart needs its point in the alt text and its numbers in a table or text. Then headings that reflect the structure, contrast a tool can test in seconds, link text that means something alone, colour never carrying meaning by itself, and keyboard access with a focus you can see. A tool settles some of these, such as contrast and a skipped heading level. Whether alt text serves its purpose, or the focus order matches the reading order, needs your judgement.
This course runs these on itself
Checker: The accessibility checker, make check-accessibility on screen, also runs inside make test, the command that runs the course's own tests. A failure there stops the build. It was run on the second of November. It checked every page and every animation's description. It also checked sixty-five contrast pairs: pairs of text and background colours. It checked them against the part of WCAG 2.2 AA that a machine can test. That is the accessibility standard named on its last line, and the check passed. Each contrast pair is printed as a ratio beside the standard's minimum. For body text on the page it is fifteen point five seven to one, where four point five is the minimum. And its output ends with a list of the things a machine cannot decide. The first is to read every alt text against what is on screen: a script can confirm that alt text is present, not that it is accurate. That is the honest shape for this whole subject.
What this unit is not saying
Narrator: Three things this unit is not saying. First, generated material is not uniquely biased. Human-written material carries distributions too, and biased, unrepresentative and inaccessible material long predates AI, in textbooks, stock images and forms. What is different is scale and invisibility: generation reproduces what it was trained on at a speed and volume no human author matches, and its output often arrives without provenance. Second, a varied output is not a fair one. Varying an output changes the output. Whether the task was appropriate is a separate question, and it is the one to ask first when the task is about people. And it is not asking you to solve this. It is asking you to notice it, check what can be checked, and escalate what cannot.
Next
The activity asks you to run the comparison three times over, in both orders, and then write what it established and what it did not. The second sentence is the harder one, and it is the one that matters. Then come who is missing, the checkable part, and the two checkpoints.
Closing, 20 seconds, no speech. A candidate record on the left, a check against an independent reference in the middle, and a bracketed claim on the right fade in side by side. Below them: Continue with the course material, ai.qyber.dev/edu-ai-literacy-ada-verification/, Course playlist: youtube.com/playlist?list=PLPsBWTfgvakM, All links are also in the description, and Keep your record. Check the next claim.