Checkpoints and Rollback

Every graph-state transition persists. When a worker crashes, its lease expires and another worker picks up at the latest checkpoint. Cloudflare Durable Objects hold state across hours or weeks. Propose-then-commit (Lesson 15) defines a rollback plan per action. Post-action verification closes the loop. EU AI Act Article 14 makes effective human oversight mandatory for high-risk systems — in practice this means checkpoints must be queryable, rollbacks must be rehearsed, and the audit trail must survive a deploy. The sharp failure mode: without idempotency keys and precondition checks, a retry after a transient failure can double-execute an already-approved action. Post-action verification is what catches it.

Learn 1 hr Python (stdlib, checkpoint and rollback state machine)

Prerequisites: Phase 15 · 12 (Durable execution), Phase 15 · 15 (Propose-then-commit)

#The Problem

Durable execution (Lesson 12) makes a crashed agent resumable. Propose-then-commit (Lesson 15) makes an approved action auditable. This lesson joins them: what happens when an approved action executes partially, crashes, and resumes? When does the rollback run, and against what state?

Real systems wire this up differently:

  • LangGraph checkpoints every graph-state transition to PostgreSQL. On worker crash, the lease releases and another worker resumes at the latest checkpoint. Workflows pause on interrupt(), which itself persists.
  • Cloudflare Durable Objects hold per-key state across hours or weeks. Co-locate the computation with the storage for the approved action.
  • Microsoft Agent Framework exposes Checkpoint primitives in the workflow API; replay plus idempotency covers retries.

In every case, the combination that actually works is: idempotency key (prevents double-execute) + precondition check (state is still what we approved against) + post-action verify (the side effect actually happened) + rollback on verify-fail.

Full Access lesson

Keep going. The rest of this lesson is one step away.

Unlock all 503 lessons, video, code, quizzes and every future update. Pay in your own currency with Paystack.

The text and code of this lesson are based on AI Engineering from Scratch by Rohit Ghumare, MIT licensed.