Backpropagation from Scratch

Backpropagation is the algorithm that makes learning possible. Without it, neural networks are just expensive random number generators.

Build 2 hr Python

Prerequisites: Lesson 03.02 (Multi-Layer Networks)

#Learning Objectives

Implement a Value-based autograd engine that builds a computational graph and computes gradients via topological sort
Derive the backward pass for addition, multiplication, and sigmoid using the chain rule
Train a multi-layer network on XOR and circle classification using only your from-scratch backpropagation engine
Identify the vanishing gradient problem in deep sigmoid networks and explain why gradients shrink exponentially

Full Access lesson

Unlock all 503 lessons, video, code, quizzes and every future update. Pay in your own currency with Paystack.

The text and code of this lesson are based on AI Engineering from Scratch by Rohit Ghumare, MIT licensed.