Watch it learn
We trained the model again with the same recipe and saved it 17 times along the way. Slide back to the start, where it knows nothing, and forward to the end.
Loss 1.85
How surprised the model is by the next token, on text it never trained on. Lower is better.
A story it writes
Once upon a time, in a small town, there was a girl named Sue. Sue liked to play with her ball. One sunny day, she went outside to play
Its guesses after "Lily wanted to play with her"
Using what came earlier in the text
- Remembering Fluffy from an earlier sentence, 82%
- Copying the made-up word Zog, 18%
Heads that look for what came next last time
A head that spreads its weight evenly scores about 0.02. Shaded cells score above that.
| Head 1 | Head 2 | Head 3 | Head 4 | |
|---|---|---|---|---|
| Layer 1 | 0.02 | 0.02 | 0.01 | 0.01 |
| Layer 2 | 0.00 | 0.00 | 0.00 | 0.00 |
| Layer 3 | 0.00 | 0.00 | 0.00 | 0.00 |
| Layer 4 | 0.02 | 0.01 | 0.03 | 0.06 |
A slider moves through saved copies of the model during training. For each one it shows the loss on a chart, a story the model writes, its guesses after a sentence, how well it uses earlier text, and a table of how much each attention head copies.
What changes
Training shows the model a batch of text, measures how surprised it was by each next token, and nudges every number a little to be less surprised next time. That surprise is theloss. At the start the model guesses blindly among 4,096 tokens, a loss of8.33. By the end it is 1.85.
The story shows the order things are learned in. First common tokens like "." and "the", then words that go together, then whole sentences about a boy named Tim. Using what came earlier in the text comes later, and fast. Remembering Fluffy from the first sentence is under 1% after 750 steps, 19% by 1,000, and past half by 3,000.
Heads that copy
Some attention heads learn a simple trick. When a token shows up again, they look at the token that came right after it last time, and pass that along as a guess. These are called induction heads. In models of many sizes, they appear at the same point in training as a sudden jump in using earlier text (Olsson et al., 2022).
The table measures this on random tokens repeated twice. A head that spreads its weight evenly scores about 0.02. In our small model the best head, layer 4 head 4, ends at 0.06, about 3 times that. Random tokens are nothing like the stories it learned from, so the skills above show the copying more clearly.
Skills can also fade. Copying the made-up word Zog peaks at 52% after 10,000 steps, and is back to 18% at the end.
This is a second run of the recipe in the chapters, so its last copy is close to the model the rest of the site uses, but not the same file.