See inside a language model

A real model is guessing the next word. Tap a guess to pick it yourself.

Once upon a time, there was a
  • little39%
  • big12%
  • boy4%
  • small2%
  • girl2%
A tiny model trained on short stories.
Learn from scratchNo math needed. Start with a story and go deeper when you like.The whole modelThe diagram from the paper. Tap a part for its sizes, formula and PyTorch code.Quick reviewFormulas, shapes and code for a refresher.PlaygroundType your own sentence and look inside the model.