part 3 boss 👾
Part 3 Boss: Fix the RAG Pipeline
the hook
A RAG pipeline is returning garbage answers. A hardcoded chunk size and a backwards similarity sort are hiding in plain sight — find both, fix both.
the idea
Someone built a RAG pipeline. It's returning garbage answers. Your job: find out why, using the exact same debugging instinct from boss 1 — read the code, don't just stare at the output.
Two bugs are hiding here: one in how the document gets chunked, one in how the 'best' chunks get picked. Neither crashes. Both quietly return the wrong thing — the most dangerous kind of bug.
Fix both, and the pipeline you built in ch15 (chunk → embed → retrieve → generate) works exactly as designed again.
playground — your hands now
what: find and fix 2 bugs in a broken RAG pipeline — a hardcoded chunk size and a backwards similarity sort.
why: both bugs are 'silent' — no crash, no error, just confidently wrong retrieval. that's the hardest kind of bug to catch.
how: run each cell, notice the wrong output, use hints if stuck, fix it, run again.
start here → run each cell first — the wrong output is your first clue.
bug 1 · the chunker
what: find and fix the bug in this chunking function.
why: a chunker that silently ignores its own chunk_size argument produces tiny, context-starved chunks no matter what the caller asks for — exactly the failure ch15's challenge warned about, but hidden.
how: run it, compare the chunk lengths to the chunk_size you passed in, then dig into the slicing.
this should produce chunks of length 30. run it — check the actual lengths. stuck? there's a hint button.
bug 2 · the retrieval sort
what: find and fix the bug in this top-k retrieval function.
why: this exact function decides which chunks get stuffed into the prompt — sorted backwards, it hands the model the LEAST relevant context every single time.
how: run it, notice the top result has the LOWEST score instead of the highest, then fix the sort direction.
this should return the 3 HIGHEST-scoring chunks first. run it — is it? stuck? there's a hint button.
try this
After fixing both, think back to ch15's chunk-size slider challenge — notice how THIS bug (hardcoded 20) is a sneakier version of the same failure: tiny chunks lose context. But at least the slider version was honest about what it was doing. A hardcoded override lies about it.
checkpoint
1. Why is a 'silent' bug (wrong output, no crash) often more dangerous than a bug that crashes?
2. What does sorting by score ASCENDING instead of DESCENDING do to a top-k retrieval function?
recap card — keep this
silent bugs — wrong output, no crash — are the most dangerous kind. they ship, and nobody notices until it's too late.