Transformer system layers

Tokens
“capital”, “Australia”, “is”
Attention
Combines context into representations.
Next-token scores
Ranks possible continuations.
Deployed system
Base transformer + post-training, with optional retrieval and tools.

Attention combines context. It is not a database lookup, a citation, or a truth test.

Transformer and system layers The Transformer in one picture A deployed assistant has layers tokens context-dependent representations attention The capital of Australia is h1 h2 h3 h4 h5 next-token scores Base transformer Post-training Retrieval (optional) Tools (optional) Attention combines context. It is not a database lookup, a citation, or a truth test.