Speculative Decoding and EAGLE-3

Phase 7 · Lesson 16 proved the math: the Leviathan rejection rule preserves the verifier's distribution exactly. This lesson is the training-stack view of 2026 production speculative decoding. EAGLE-3 turned the draft model from a cheap approximation into a purpose-built tiny network trained on the verifier's own hidden states, then added a training-time test loop that aligns its train and inference distributions. Result: 3× to 6.5× end-to-end speedup, accepted per-token rates above 0.9 on chat, no distributional tradeoff. Every production inference stack in 2026 ships it by default.

Build 1.3 hr Python (stdlib)

Prerequisites: Phase 7 · 16 (speculative decoding math), Phase 10 · 12 (inference optimization)

#Learning Objectives

State the Leviathan theorem in one sentence and prove that the speculative loop produces samples identically distributed to the verifier.
Walk the two-year progression from vanilla spec-decoding (Leviathan 2023) through EAGLE, EAGLE-2, and EAGLE-3 and name the exact limitation each step removed.
Compute expected speedup from acceptance rate α and draft-to-verifier cost ratio c, and choose the optimal draft length N for each regime.
Implement the full speculative loop from scratch: draft, verify, reject-sample from the residual, roll the KV cache back on rejection, emit the bonus token on full acceptance.

#Learning Objectives

Keep going. The rest of this lesson is one step away.