AI Engineering/Phase 17/Lesson 14

Prompt Caching and Semantic Caching Economics

**Pricing snapshot dated 2026-04.** Numeric claims below reflect vendor rate cards captured at this lesson's publication; verify against the linked docs before quoting them downstream.

Learn 1 hr Python (stdlib, toy two-layer cache simulator)

Prerequisites: Phase 17 · 04 (vLLM Serving Internals), Phase 17 · 06 (SGLang RadixAttention)

#Learning Objectives

Distinguish L2 prompt/prefix caching (KV reuse at provider) from L1 semantic caching (LLM bypass on similar prompts).
Explain Anthropic's cache_control explicit marking and the two TTL options (5-min vs 1-hour) with their price multipliers.
Compute expected monthly savings given hit rate, prompt/response mix, and token prices.
Name the parallelization anti-pattern that inflates bills by 5-10x and the dynamic-content anti-pattern that collapses hit rate.

Full Access lesson

Keep going. The rest of this lesson is one step away.

Unlock all 503 lessons, video, code, quizzes and every future update. Pay in your own currency with Paystack.

See plans Sign in

The text and code of this lesson are based on AI Engineering from Scratch by Rohit Ghumare, MIT licensed.