chapter 23 · part 4
The PyTorch bridge
the hook
Your next 2-3 months, mapped week by week — from this app to training real models on real GPUs.
the idea
Everything in this app was hand-rolled — no PyTorch, no TensorFlow, real math but built from scratch so you could see every gear turning. That was deliberate: understanding beats magic.
But real ML work happens in a framework, and PyTorch is the one worth learning. The good news: you already know the concepts. `net.forward()`, `net.backward()`, `net.step()` from ch4-9 — that's literally the same shape as PyTorch's `model(x)`, `loss.backward()`, `optimizer.step()`.
Below: 4 side-by-side comparisons, your actual code from earlier chapters next to the equivalent real PyTorch. Same ideas, different syntax, way less code to write yourself.
And a 12-week roadmap — from PyTorch basics to finetuning an open model on a rented GPU. Check off weeks as you go; it's saved in your browser.
playground — your hands now
what: compare your own code from earlier chapters to the equivalent real PyTorch, then check off a 12-week roadmap.
why: this is the bridge from 'I understand how it works' to 'I can build real models with real tools' — the whole point of days 1-4.
how: read the 4 comparisons, then check off roadmap weeks as you commit to them.
start here → read the 4 comparisons, then check off roadmap weeks as you commit to them.
your code vs pytorch
a neuron's forward pass — from ch4 / engine/tinynn.js
your code
forward(x) {
this.z = this.w.map((row, i) =>
row.reduce((sum, wij, j) => sum + wij * x[j], this.b[i])
);
this.a = this.z.map(tanh);
return this.a;
}pytorch
import torch.nn as nn layer = nn.Linear(in_features, out_features) a = torch.tanh(layer(x))
a gradient descent step — from ch8
your code
const grad = fPrime(prevX); const nextX = prevX - lr * grad;
pytorch
optimizer = torch.optim.SGD([x], lr=0.1) loss = f(x) optimizer.zero_grad() loss.backward() optimizer.step()
scaled dot-product attention — from ch5 / ch6
your code
function attention(x) {
const scale = Math.sqrt(DIM);
return x.map((qi) => {
const weights = softmax(x.map((kj) => dot(qi, kj) / scale));
return weightedSum(x, weights);
});
}pytorch
import torch.nn.functional as F scores = Q @ K.transpose(-2, -1) / (d_k ** 0.5) weights = F.softmax(scores, dim=-1) output = weights @ V
one training step — from ch9 / lib/charmodel.js
your code
const out = net.forward(x); const probs = softmax(out, 1); const grad = probs.map((p, i) => p - onehot[i]); net.backward(grad); net.step(lr);
pytorch
optimizer.zero_grad() logits = model(x) loss = F.cross_entropy(logits, target) loss.backward() optimizer.step()
12-week roadmap (0 / 12 checked)
try this
Pick ONE comparison below and actually run the PyTorch version — install torch, paste it into a script, run it. That's the fastest way to confirm the bridge is real, not just a diagram.
checkpoint
1. Why did this app build everything from scratch instead of using PyTorch from day one?
2. What's the real relationship between your `net.forward()` / `backward()` / `step()` from ch4-9 and PyTorch's equivalents?
recap card — keep this
weeks 1-2 pytorch basics · 3-4 rebuild our tiny GPT · 5-8 nanoGPT on a rented GPU · 9-12 LoRA-finetune an open model.