AI Engineering/Phase 14/Lesson 05

Self-Refine and CRITIC: Iterative Output Improvement

Self-Refine (Madaan et al., 2023) uses one LLM in three roles — generate, feedback, refine — in a loop. Average gain: +20 absolute on 7 tasks. CRITIC (Gou et al., 2023) hardens the feedback step by routing verification through external tools. In 2026 this pattern ships in every framework as "evaluator-optimizer" (Anthropic) or a guardrail loop (OpenAI Agents SDK).

Build 1 hr Python (stdlib)

Prerequisites: Phase 14 · 01 (Agent Loop), Phase 14 · 03 (Reflexion)

#Learning Objectives

State Self-Refine's three prompts (generate, feedback, refine) and explain why history matters for the refine prompt.
Explain CRITIC's critical insight: LLMs are unreliable at self-verification without external grounding.
Implement a stdlib Self-Refine loop with history and an optional external verifier.
Map this pattern to Anthropic's "evaluator-optimizer" workflow and OpenAI Agents SDK's output guardrails.

Full Access lesson

Keep going. The rest of this lesson is one step away.

Unlock all 503 lessons, video, code, quizzes and every future update. Pay in your own currency with Paystack.

See plans Sign in

The text and code of this lesson are based on AI Engineering from Scratch by Rohit Ghumare, MIT licensed.