AI Engineering/Phase 19/Lesson 53

Result Evaluator

The runner produced numbers. The evaluator decides whether those numbers are an improvement, a regression, or noise. Build the verdict path that turns metrics into a one line conclusion.

Build 1.5 hr Python

Prerequisites: Phase 19 Track A lessons 20-29

#Learning Objectives

Compare a candidate run against a baseline using direction aware improvement and a fixed threshold.
Run a paired t test from scratch over per seed metrics and read the resulting p value.
Normalise log scaled metrics so a downstream report can blend them with linear metrics.
Emit a per hypothesis verdict that the orchestrator can attach to the queue from lesson fifty.
Keep every step pure so the same inputs always produce the same verdict.

Full Access lesson

Keep going. The rest of this lesson is one step away.

Unlock all 503 lessons, video, code, quizzes and every future update. Pay in your own currency with Paystack.

See plans Sign in

The text and code of this lesson are based on AI Engineering from Scratch by Rohit Ghumare, MIT licensed.