Model Routing as a Cost-Reduction Primitive

A dynamic broker evaluates every request (task type, token length, embedding similarity, confidence) and sends simple queries to a cheap model, escalating complex ones to a frontier model. Also called model cascading. Production case studies show 20-60% cost reduction at iso-quality across US/UK/EU deployments; a 30% routing efficiency improvement on high-volume SaaS turns into six-figure annual savings. The 2026 context is that LLM inference prices dropped ~10x per year — a GPT-4-class token went from $20/M to ~$0.40/M from late 2022 to 2026. Most of the drop is better serving stacks (Phase 17 · 04-09), not hardware. Routing is how you convert that price drop into margin without product regression. The failure mode is cheap-model drift: the route pushes 40% to a weaker model, quality drops 3-5% on reasoning tasks, no one notices for a quarter. Gate routes by online quality metrics, not just offline eval sets.

Learn 1 hr Python (stdlib, toy cascading router simulator)

Prerequisites: Phase 17 · 01 (Managed LLM Platforms), Phase 17 · 19 (AI Gateways)

#Learning Objectives

Explain model cascading: cheap-first with confidence check, escalate on low confidence.
Enumerate the four routing signals (task classification, prompt length, embedding similarity to known-hard set, self-confidence from first-pass).
Compute expected blended cost at target routing split and quality loss tolerance.
Name the drift-monitoring metric (online quality gate) that catches cheap-model creep.

#Learning Objectives

Keep going. The rest of this lesson is one step away.