Unit 15 activity: Bias, representation and accessibility

Time: about 20 minutes. Read the unit page first.

Part 1 — the one-attribute comparison (12 minutes)

Pick a prompt that asks for something about a person or a role — a short character description, a worked example about a professional, a scenario involving a named individual.

Use a fictional person throughout. Do not build this on a real name: you would be inferring things about someone who did not ask to be in your exercise, and the exercise works just as well without it.

1. Write the two versions of the prompt first, differing in exactly one attribute — a name, a pronoun, a country, a language. Decide before you run anything which features you are going to record, so that you are not choosing them after seeing what came back.
2. Run three pairs, each in separate fresh conversations (Unit 5 — the same conversation is not a second trial). Alternate which version you run first, because order is one more thing that can differ.
3. Record, for each of your predefined features, how many of the three runs on each side showed it. Specifics rather than impressions: which nouns, which adjectives, which details appeared, and what was assumed without being asked.

If you have no AI access, use the tally from the reading, an illustration rather than a real run. Prompts A and B, three runs each. A different occupation: 3 of 3 on side A, 0 of 3 on side B. Family mentioned: 1 of 3 on each side. Write the two sentences about that. For Part 2 use any short description of a person that someone else wrote.

Six runs rather than two, because two fresh conversations differ in two ways — the attribute you changed and the system's own run-to-run variation — and a single pair cannot tell you which one you are looking at. Three each is not a study. It is the smallest number that lets you see whether a difference persists, and seeing that is the point of the exercise.

Then write two sentences, and be careful about both:

What this comparison established: …

What it did not establish: …

The second sentence is the harder one and the one that matters. Six runs are six runs, and with so few even a clean pattern could be chance; more runs would test it. If a feature showed up on one side every time and never on the other, you have a pattern worth reporting; if it showed up twice on one side and once on the other, you have watched a system vary. Either way you have not measured a rate, you have not shown that no difference exists elsewhere, and you have not found the cause.

Report it as what it is: on this system, on this date, across three pairs, these features appeared this often. That sentence is defensible. "The model is biased against X" is not, on this evidence, and the defence you write in Unit 17 turns on the difference.

If the outputs were near-identical throughout, say so. That is a result, with the same limits attached.

Part 2 — who is missing (3 minutes)

Take one of the outputs and answer three questions.

The middle row is the point. Absences are easy to miss when your reference omits the same groups, which is why this starts with a prompt rather than a procedure. Where you can, also compare against a list of relevant groups assembled independently of the output.

Part 3 — the checkable part (5 minutes)

Take a piece of generated material you might actually publish or hand in: a figure with a caption, a page with headings, a slide, a short document.

If you have no AI access, use a page or slide a person made, and write both versions of the alt text yourself, one saying what is in the image, one saying what it is for.

Run the checks. A tool settles contrast and a skipped heading level; the others need your judgement:

• [ ] Alt text on every image. Does it say what the image conveys for the purpose you are using it for, not just what is in it?
• [ ] Headings in order, with no level skipped where the structure does not skip.
• [ ] Contrast meets the standard. Use a contrast checker; it takes seconds.
• [ ] Link text that means something out of context.
• [ ] No meaning carried by colour alone.

Record what failed. Then rewrite one piece of alt text yourself and note the difference between the two versions — the generated one will usually describe the contents, and yours will describe the function.

Now take the checkpoints

cp15-what-a-check-shows is about Part 1's second sentence. cp15-alt-text is about Part 3.

python3 selfcheck.py run cp15-what-a-check-shows  # or in the browser
python3 selfcheck.py run cp15-alt-text  # or in the browser

Run the Python route from The verification lab and self-check, or use the browser self-check.

Optional extensions

Optional. These sit outside the core study time, are not required for the checkpoints, and nothing later in the course depends on them.

Test the thin half (about 6 minutes). Ask for something specific about a smaller place, a smaller language, or a less-documented practice that you happen to know well. Then check it. This is the half of the unit that gets less attention and it is often where the clearest errors are — and you are one of the few people positioned to catch them, which is itself the finding.

Read this course's own accessibility output (about 4 minutes). Run make check-accessibility and read what it prints, including the list at the end of things it cannot decide. Compare that list with the checks you ran in Part 3. A pass from a machine is not the same as a claim that the page meets the standard, and the script says so itself.