chapter 15 · part 3
RAG — build it yourself ★
the hook
Ask a model about your college's fees. It hallucinates. You're about to fix that, permanently.
the idea
This is the big one — a real RAG (Retrieval-Augmented Generation) pipeline, built stage by stage, by you. Same idea that fixes hallucination in every real product: don't trust the model's memory, hand it the actual source material right before it answers.
Step one: chop your document into overlapping chunks — small enough to be specific, overlapping enough that an idea split across a chunk boundary doesn't get orphaned.
Step two: turn every chunk into a point in space (an embedding) — exactly like ch3, except now it's real chunks of YOUR document, not hand-placed toy words.
Step three: when a question comes in, embed IT too, then find whichever chunks are closest by cosine similarity — that's retrieval. Stuff those chunks into the prompt, then ask the model to answer using ONLY that context — that's augmented generation.
Quick clarifier, because this trips people up: the embedding model that just built your map (MiniLM, or the local fallback) is a COMPLETELY SEPARATE model from the LLM answering your question at the end. One turns text into searchable points in space. The other predicts next tokens. RAG is just the plumbing connecting the two — they are not the same model wearing two hats.
playground — your hands now
what: build a full RAG pipeline in 5 stages: paste a document, chunk it, embed it, compute similarity, then ask a real question and get a real, context-grounded answer.
why: this exact pipeline — chunk, embed, retrieve, stuff into prompt — is how every production RAG system works, from customer support bots to your college's chatbot.
how: work top to bottom through the 5 stages. drag the chunk-size slider to see retrieval literally break.
start here → work top to bottom — paste a document, chunk it, embed it, then ask a question.
stage 1 · the document
1147 characters
stage 2 · write the chunker
what: write a fixed-size chunking function with overlap, using string slicing.
why: this exact function — chop text into overlapping windows — is step one of every RAG pipeline, including the live one below.
how: slice text[i:i+chunk_size], then advance i by (chunk_size - overlap) each loop.
predict first: with overlap > 0, will consecutive chunks share any text?
given text, chunk_size, and overlap, return a list of overlapping chunks. stuck? there's a hint button.
chunk-size slider (breaks retrieval on purpose)
10 chunks at this size
stage 3 · embed
stage 4 · write cosine similarity
what: write cosine similarity from scratch — dot product over the product of magnitudes.
why: this is exactly how the pipeline above decides which chunks are 'closest' to your question.
how: compute the dot product, divide by (magnitude(a) * magnitude(b)).
predict first: if chunk_vec and query_vec point in a very similar direction, will cosine similarity be closer to 0 or closer to 1?
given two vectors, compute their cosine similarity. stuck? there's a hint button.
stage 5 · ask
predict first: will the top-3 retrieved chunks actually be about hostel fees, or something else?
embed the chunks first (stage 3).
try this
Set the chunk size absurdly small (like 20 characters) and ask your question again. Watch retrieval get worse — tiny chunks lose context. Then set it absurdly large (the whole document as 1 chunk) and watch a different failure: now EVERY question 'matches' because there's only one thing to match against. The sweet spot is real, and you just found both edges of it.
checkpoint
1. What are the two separate things that happen in retrieval-augmented generation?
2. Is the embedding model that powers search/RAG the same model that generates the final answer?
recap card — keep this
chunk docs → embed → store → retrieve nearest → stuff into prompt. that's RAG, and you built it.