vLLM Serving Internals: PagedAttention, Continuous Batching, Chunked Prefill

vLLM's dominance in 2026 rests on three compounding defaults, not a single trick. PagedAttention is always on. Continuous batching injects new requests into the active batch between decode iterations. Chunked prefill slices long prompts so decode tokens never starve. Turn all three on and a Llama 3.3 70B FP8 on one H100 SXM5 pushes 2,200-2,400 tok/s at 128 concurrent — roughly 25% above vLLM's own default and 3-4x a naive PyTorch loop. This lesson reads the scheduler and attention kernel at a level you can diagram, and ends with a toy continuous batcher in `code/main.py` that schedules prefill and decode the way vLLM does.

Learn 1.3 hr Python (stdlib, toy continuous batching scheduler)

Prerequisites: Phase 17 · 01 (Model Serving), Phase 11 (LLM Engineering)

#Learning Objectives

Explain PagedAttention as a KV cache allocator: blocks, block tables, and why fragmentation stays under 4% at production load.
Diagram continuous batching at the iteration level: how finished sequences leave the batch and new ones join without draining.
Describe chunked prefill in one sentence and name which latency metric it protects (hint: it is TTFT tail, not mean throughput).
Name the 2026 vLLM v0.18.0 gotcha that bites teams enabling every optimization at once.

#Learning Objectives

Keep going. The rest of this lesson is one step away.