Sign inStart free
AI Engineering/Phase 03
Core

Deep Learning Core

Backpropagation, optimizers and your own neural network library.

13 lessons ~18.8 hr
  1. The PerceptronBuild · Free · 1 hThe perceptron is the atom of neural networks. Split it open and you find weights, a bias, and a decision.Open
  2. Multi-Layer Networks and Forward PassBuild · 1 h 30 minOne neuron draws a line. Stack them, and you can draw anything.Needs Full Access
  3. Backpropagation from ScratchBuild · 2 hBackpropagation is the algorithm that makes learning possible. Without it, neural networks are just expensive random number generators.Needs Full Access
  4. Activation FunctionsBuild · 1 h 15 minWithout nonlinearity, your 100-layer network is a fancy matrix multiply. Activations are the gates that let neural networks think in curves.Needs Full Access
  5. Loss FunctionsBuild · 1 h 15 minYour network makes a prediction. The ground truth says otherwise. How wrong is it? That number is the loss. Pick the wrong loss function and your model optimizes for the wrong thing entirely.Needs Full Access
  6. OptimizersBuild · 1 h 15 minGradient descent tells you which direction to move. It says nothing about how far or how fast. SGD is a compass. Adam is GPS with traffic data.Needs Full Access
  7. RegularizationBuild · 1 h 15 minYour model gets 99% on training data and 60% on test data. It memorized instead of learning. Regularization is the tax you impose on complexity to force generalization.Needs Full Access
  8. Weight Initialization and Training StabilityBuild · 1 h 30 minInitialize wrong and training never starts. Initialize right and 50 layers train as smoothly as 3.Needs Full Access
  9. Learning Rate Schedules and WarmupBuild · 1 h 30 minThe learning rate is the single most important hyperparameter. Not the architecture. Not the dataset size. Not the activation function. The learning rate. If you tune nothing else, tune this.Needs Full Access
  10. Build Your Own Mini FrameworkBuild · 2 hYou have built neurons, layers, networks, backprop, activations, loss functions, optimizers, regularization, initialization, and LR schedules. All as separate pieces. Now wire them together into a framework. Not PyTorch. Not TensorFlow. Yours.Needs Full Access
  11. Introduction to PyTorchBuild · 1 h 15 minYou built the engine from pistons and crankshafts. Now learn the one everyone actually drives.Needs Full Access
  12. Introduction to JAXBuild · 1 h 30 minPyTorch mutates tensors. TensorFlow builds graphs. JAX compiles pure functions. That last one changes how you think about deep learning.Needs Full Access
  13. Debugging Neural NetworksBuild · 1 h 30 minYour network compiled. It ran. It produced a number. The number is wrong and nothing crashed. Welcome to the hardest kind of debugging -- the kind where there is no error message.Needs Full Access