AI Engineering/Phase 03/Lesson 05

Loss Functions

Your network makes a prediction. The ground truth says otherwise. How wrong is it? That number is the loss. Pick the wrong loss function and your model optimizes for the wrong thing entirely.

Build 1.3 hr Python

Prerequisites: Lesson 03.04 (Activation Functions)

#Learning Objectives

Implement MSE, binary cross-entropy, categorical cross-entropy, and contrastive loss (InfoNCE) from scratch with their gradients
Explain why MSE fails for classification by demonstrating the "predict 0.5 for everything" failure mode
Apply label smoothing to cross-entropy and describe how it prevents overconfident predictions
Choose the correct loss function for regression, binary classification, multi-class classification, and embedding learning tasks

Full Access lesson

Keep going. The rest of this lesson is one step away.

Unlock all 503 lessons, video, code, quizzes and every future update. Pay in your own currency with Paystack.

See plans Sign in

The text and code of this lesson are based on AI Engineering from Scratch by Rohit Ghumare, MIT licensed.