chapter 18 · part 3
Finetuning in practice (LoRA)
the hook
Full finetuning rewrites billions of weights. LoRA sneaks in tiny adapters instead. 100x cheaper.
the idea
Full finetuning means updating every single one of a model's weights — for a model with billions of parameters, that's an enormous amount of numbers to store, move, and retrain, even for a small behavior change.
LoRA's trick: freeze the ENTIRE original model, and instead train two tiny matrices whose product approximates the update you actually wanted. The original weights never move — only the small add-on does.
Why does this work? Most useful weight updates turn out to be 'low rank' — they don't need the full richness of a giant matrix to capture the change. A skinny approximation captures almost all of the value at a tiny fraction of the parameter count.
Below: watch a frozen weight matrix stay frozen while two small trainable matrices do all the work, and drag the rank slider to see the real parameter-count math for yourself.
playground — your hands now
what: watch a frozen weight matrix stay frozen while two small trainable matrices do the actual work, then compare real parameter counts.
why: LoRA is why you can finetune a giant model on a laptop instead of a data center — this is the actual mechanism, not marketing.
how: drag the matrix size and rank sliders and watch the real parameter math update below.
start here → drag the rank slider and watch the real parameter math change below.
🔒 frozen weight matrix (never updated)
1024 × 1024 = 1,048,576 params
✓ trainable adapters (A × B)
2 × 1024 × 8 = 16,384 params
LoRA trains 16,384 parameters instead of 1,048,576 — just 1.56% of full finetuning, and the frozen matrix above never moves.
finetune vs RAG — decide for your case
does your use case need the model to know NEW FACTS it wasn't trained on (like your company's docs)?
bonus · open vs closed models
what: compare open-weight models (Llama, Gemma, Mistral) against closed API models (GPT, Claude, Gemini).
why: this decision shapes cost, control, and what you're even allowed to do with a model — pick wrong and you'll feel it later.
how: scan the table, then use the decision tree below for YOUR specific case.
| open (Llama, Gemma, Mistral) | closed (GPT, Claude, Gemini) | |
|---|---|---|
| weights | download & own them (Llama, Gemma, Mistral) | never see them (GPT, Claude, Gemini) |
| cost model | pay once for hosting/compute | pay per token, forever |
| finetuning | fully free — LoRA, full finetune, anything | limited to what the provider's API allows |
| getting started | need your own GPU or cloud instance | an API key and you're done in minutes |
| quality ceiling | improving fast, usually a bit behind frontier | usually the most capable models available |
try this
Set the matrix size to something realistic (like 4096×4096, roughly one attention weight matrix in a mid-size model) and the rank to 8. Look at the percentage of parameters LoRA actually trains. Then imagine that ratio repeated across hundreds of weight matrices in a real model — that's the whole story of why LoRA is cheap.
checkpoint
1. What does LoRA actually freeze, and what does it train?
2. What's the main practical tradeoff between open and closed models?
recap card — keep this
LoRA = freeze the model, train small add-on matrices. finetune behavior, not knowledge. RAG for facts.