LLM Lab — build your own AI, by hand

the idea

An agent is a model wrapped in a loop: think, pick a tool, observe what happened, repeat — until it decides it's done. No new model, no new magic. Same next-token prediction, just given permission to act between guesses.

The 'tools' are just functions — a calculator, a search function, a notepad. The model doesn't run them itself; it asks for one by name (and arguments), your code actually runs it, and the result gets fed back in as new context.

Real APIs (OpenAI, Gemini, Claude) formalize this with a strict JSON contract: you describe each tool's name, description, and expected arguments up front, and the model's 'I want to call a tool' response comes back in a matching structured format — never free text, always parseable JSON.

Below: 3 real local tools (calculator, search, notepad) and an agent loop that actually calls them. With a Gemini key, the model itself drives the loop. Without one, a scripted replay shows you the exact same loop shape.

playground — your hands now

what: watch an agent loop through think → tool → observe using 3 real local tools, either live (Gemini) or scripted.

why: 'agent' sounds mystical until you see the loop — it's just a model, 3 functions, and a while loop feeding results back in.

how: with a Gemini key: type a request and watch it drive the loop live. without one: step through a scripted example.

start here → step through the scripted replay — or add a gemini key via ⚙ to drive it live.

scripted replay (no Gemini key needed)

"what's 15% of 240, and what does RAG stand for? save both to my notes."

no Gemini key set, so this is a fixed script — but every step actually runs the real local tools (the calculator genuinely computes 240 × 0.15, the search genuinely looks up "RAG"). add a key via the ⚙ icon to let a real model drive the loop instead.

bonus · the structured tool-calling contract

how a real API describes a tool:

{
  "name": "calculator",
  "description": "Evaluates a basic arithmetic expression.",
  "parameters": {
    "type": "object",
    "properties": {
      "expression": { "type": "string" }
    },
    "required": ["expression"]
  }
}

how the model asks to call it:

{
  "tool_calls": [
    {
      "name": "calculator",
      "arguments": { "expression": "240 * 0.15" }
    }
  ]
}

what: write the tool-picker if/else logic in python.

why: this exact decision — which tool does this request need? — is what the JSON contract above formalizes for real APIs.

how: check for math trigger words first, then lookup words, and fall back to notepad.

predict first: what should pick_tool("what's 12 times 8?") return?

finish pick_tool() so it returns 'calculator', 'search', or 'notepad' for each request. stuck? there's a hint button.

try this

Ask (or read, in scripted mode) for something that needs 2 different tools in sequence — a calculation AND a lookup. Notice the model has to decide the ORDER itself, and has to correctly read the observation from tool #1 before deciding what tool #2 needs.

checkpoint

1. In an agent loop, who actually executes the tool the model asks for?

2. Why do real tool-calling APIs demand strict JSON instead of letting the model just describe what it wants in plain English?

recap card — keep this

agent = LLM in a loop: think → pick tool → observe → repeat until done.

next chapter → Finetuning in practice (LoRA)

Agents & tool use