chapter 01 · part 1
What is a language model?
the hook
Every AI you've ever talked to does exactly one thing: guess the next word. That's it. Seriously.
the idea
Here's the secret that sounds too simple to be true: a language model is a machine that looks at some text and guesses what comes next. One token at a time. That's the whole job.
"The capital of India is ___" → the model has seen enough text to know "New" is very likely, then "Delhi". String enough good guesses together and you get essays, code, conversations — everything you've ever seen an AI do.
But before the model can guess anything, text has to become something a machine can compute with. Machines don't read letters — they read numbers. So step zero of every LLM is chopping text into pieces called tokens, and giving each piece an ID number.
Try it yourself below. This is the exact first step your message goes through every single time you talk to Claude or GPT.
playground — your hands now
what: type text and watch it get chopped into tokens, 3 different ways.
why: this is the literal first thing that happens to your message before any model reads it.
how: type in the box, switch modes, hover a chip to see its id.
start here → type your name below and switch between the three modes.
what the model actually receives: [14287, 12063, 48624, 37939, 3370, 20198, 3904] — just numbers. no letters. ever.
try this
Type 'strawberry' in the subword mode below and count the pieces it breaks into. Then type a Hinglish sentence — 'kya scene hai bro' — and notice which words survive whole and which get shredded. Rare words get shredded. That shredding is exactly why models miscount letters in words: they never saw the letters, only the chunks.
checkpoint
1. What does a language model fundamentally do?
2. Why does a model struggle to count the r's in 'strawberry'?
recap card — keep this
an LLM = a very good next-token guesser. everything else is built on that.