Scaffolding Is the Leverage

scaffoldingclaude-codeagentsengineering

Trimmed to bullet-point summary 2026-05-09 — original prose archived at _archive/blog-articles-pre-trim-2026-05-09/scaffolding-is-the-leverage.md. Pending rewrite in voice.

Frame

  • Karpathy: AGI 10 years out because LLMs can’t learn on the job.
  • Counter: model hasn’t been the load-bearing variable for a while. Scaffolding around it has.

What Actually Got Better

  • Claude Opus 4 → 4.7 = real delta but not the working-life delta.
  • Claude Code launched ~March 2025 as ~5x Opus multiplier. 10 months later night-and-day better.
  • Most of that delta from scaffolding (planning loops, skills, subagents, context handling, hooks, memory), not base model.
  • Karpathy is measuring naked LLM. Real question (Miessler frame): does a stitched system replace the work of a knowledge worker?

The Receipts

  • XBOW = #1 HackerOne hacker in the United States. Fully automated AI agent beating every human bug hunter on a platform that pays only for real, first-to-find, exploitable vulnerabilities. Wins via relentless / parallel / never-tires system, not via smarter underlying model.
  • Jason Haddix’s AI-driven recon stack found a live P1 within 15 min of being pointed at an admin login page. First move: add id=1 to POST → instant bypass. Move that lives in human methodology but isn’t reached for first; AI flattens cost of trying moves humans deprioritize.
  • AI finding real kernel-class bugs in production systems (XBOW work, AIxCC results, accepted reports at Google/Microsoft). Matthew Brown (Trail of Bits, second at AIxCC) when asked model-or-system: System. Every time.

Author’s Stack

  • Isidore on DAI. Sharper-when-shipped — author shipping new skill / hook / routing / subagent context tightening / planning checklist, not when Anthropic ships base-model upgrades.
  • Architecture pattern Miessler describes: separate subfolders, separate CLAUDE.md files, separate large context per subagent so heavy context stays compartmentalized.
  • Built a small internal tool in an afternoon that would have been a 2-week project a year ago. Not faster — better-scaffolded.

Dynamic Context Frontier

  • Miessler’s named missing capability: dynamic context — cheap fast pull of perfect knowledge into perfect moment for perfect decision.
  • Humans do this automatically (decades of experience folded into every small call).
  • Models still manually stuff context via retrieval / prompt tricks / skills / subagent handoffs.
  • Gap between dynamic-context model vs not is larger than gap between two frontier model generations.
  • Engineering moves to close it: chunking strategies, real-time vector stores, tiny local retrievers, skills as modular expertise packs, persistent memory files, planning loops.

Model Lab Cosplay

  • Miessler companion piece (March): never believed in custom small models / fine-tuning for enterprise tasks. Best SOTA + sharp context every time.
  • Fine-tuning locks in yesterday’s knowledge, caps reasoning, recreates at cost what the next frontier release ships free.
  • Every enterprise that played model lab → brittle specialist stale within a quarter.
  • Context is the product surface. Weights are not.

Karpathy Argument Resolution

  • AGI = naked model continuous learning via gradient flow → Karpathy wins. True and irrelevant.
  • AGI = system replaces median knowledge worker → scaffolding already closed most of the gap. Trillions of dollars against a low bar.

Wake-up

  • Most leveraged moment: the gap between bare model and well-scaffolded model is where value sits and almost nobody is building there.
  • Skills, hooks, memory, context routing, subagent orchestration, planning loops, verification layers. Pick one. Go deep.