FinOps for LLMs — Unit Economics and Multi-Tenant Attribution

Traditional FinOps breaks on LLM spend. Costs are token-transactions, not resource-uptime. Tags don't map — an API call is a transaction, not an asset. Engineering decisions (prompt design, context window, output length) are financial decisions. The 2026 playbook has three attribution dimensions to instrument on day one: per-user (`user_id`) for seat pricing and expansion, per-task (`task_id` + `route`) for product surface cost and prioritization, per-tenant (`tenant_id`) for unit economics and renewal. Four token layers — prompt, tool, memory, response — one bucket hides spend. Enforcement ladder for multi-tenant products: rate limits per tenant (2-3x expected peak, clear 429 + retry-after); daily spend cap (1.5-3x contracted ceiling; triggers rate tightening + alert); kill switches on spend z-score > 4 (auto-pause + page on-call). Attribution patterns: tag-and-aggregate, telemetry-joiner (trace-ID → billing; highest accuracy), sampling-and-extrapolation, model-based allocation, event-sourced, real-time streaming. Unit metric: cost per resolved query, cost per generated artifact — not $/M tokens. Retroactive tagging always misses; instrument at request creation.

Learn 1 hr Python (stdlib, toy cost-attribution simulator with kill switch)

Prerequisites: Phase 17 · 13 (Observability), Phase 17 · 14 (Caching)

#Learning Objectives

  • Explain why traditional FinOps (tags + tiers) breaks on LLM spend and name the three new attribution dimensions.
  • Enumerate the four token layers (prompt, tool, memory, response) and why single-bucket billing hides cost.
  • Design an enforcement ladder (rate → spend cap → kill switch) for a multi-tenant product.
  • Pick a unit metric (cost per resolved query / artifact) instead of $/M tokens.
Full Access lesson

Keep going. The rest of this lesson is one step away.

Unlock all 503 lessons, video, code, quizzes and every future update. Pay in your own currency with Paystack.

The text and code of this lesson are based on AI Engineering from Scratch by Rohit Ghumare, MIT licensed.