AI Engineering/Phase 19/Lesson 68

RAG Evaluation: Precision, Recall, MRR, nDCG, Faithfulness, Answer Relevance

If you cannot grade your retrieval and your answer at the same time, you cannot ship the system. The two are not the same metric and the same prompt fails on different axes.

Build 1.5 hr Python

Prerequisites: Phase 11 lessons 06 (RAG), 10 (evaluation); Phase 19 Track B foundations (lessons 20-29); Phase 19 lessons 64, 65, 66, 67

#Learning Objectives

Compute four retrieval metrics from gold qrels: precision@k, recall@k, MRR (mean reciprocal rank), and nDCG@k.
Compute two answer-grade metrics: faithfulness (every claim grounded in retrieved context) and answer relevance (the answer addresses the question).
Build a fixture qrels file (queries, gold doc ids, gold answer text) that the eval reads end to end.
Read the metric values to diagnose where a pipeline is failing: retrieval, ranking, generation, or grounding.

Full Access lesson

Keep going. The rest of this lesson is one step away.

Unlock all 503 lessons, video, code, quizzes and every future update. Pay in your own currency with Paystack.

See plans Sign in

The text and code of this lesson are based on AI Engineering from Scratch by Rohit Ghumare, MIT licensed.