3D Gaussian Splatting from Scratch

A scene is a cloud of millions of 3D Gaussians. Each one has a position, orientation, scale, opacity, and a colour that depends on viewing direction. Rasterise them, backprop through the rasterisation, done.

Build 1.5 hr Python

Prerequisites: Phase 4 Lesson 13 (3D Vision & NeRF), Phase 1 Lesson 12 (Tensor Operations), Phase 4 Lesson 10 (Diffusion basics optional)

#Learning Objectives

  • Explain why 3D Gaussian Splatting replaced NeRF as the production default for photorealistic 3D reconstruction in 2026
  • State the six per-Gaussian parameters (position, rotation quaternion, scale, opacity, spherical harmonics colour, optional feature) and how many floats each contributes
  • Implement a 2D Gaussian splatting rasterizer from scratch using alpha compositing, then show how the 3D case projects to the same loop
  • Use nerfstudio, gsplat, or SuperSplat to reconstruct a scene from 20-50 photos and export to the KHR_gaussian_splatting glTF extension or the OpenUSD 26.03 UsdVolParticleField3DGaussianSplat schema
Full Access lesson

Keep going. The rest of this lesson is one step away.

Unlock all 503 lessons, video, code, quizzes and every future update. Pay in your own currency with Paystack.

The text and code of this lesson are based on AI Engineering from Scratch by Rohit Ghumare, MIT licensed.