Learn

The first chapters follow one sentence through the model, and the later ones show how it is used today. Pick how you want things explained. You can change it at any step.

The whole modelA clickable map of every part, with sizes, formulas and PyTorch code.

Inside the model

  1. 0What a language model doesA language model does one thing over and over. It guesses what comes next.About 8 minutes
  2. 1TokensBefore a model can read text, it cuts the text into pieces called tokens and swaps each piece for a number.About 10 minutes
  3. 2Embeddings and positionEach token becomes a list of 128 numbers, plus a second list for where it sits in the text.About 12 minutes
  4. 3AttentionAttention lets each token gather what it needs from itself and the tokens before it.About 15 minutes
  5. 4PredictionThe last token's numbers become a score for every token, and the scores become chances to pick from.About 12 minutes

Using it today

How prompts, retrieval, tools and reasoning build on the same model.

  1. 5Context and promptsA model only reads the text in front of it, so what you put in that text, the prompt, steers every guess.About 10 minutes
  2. 6Retrieval and toolsA model only knows what it learned and what is in its window, so programs around it look things up and do sums, and put the results in the text.About 10 minutes
  3. 7Thinking step by stepA model that writes out its steps before the answer gets each step into its window, and models trained to do this solve much harder problems.About 8 minutes