Sign inStart free
AI Engineering/Phase 04/Lesson 27

Multi-Object Tracking & Video Memory

Tracking is detection plus association. Detect every frame. Match this frame's detections to last frame's tracks by ID.

Build 1 hr Python

Prerequisites: Phase 4 Lesson 06 (YOLO Detection), Phase 4 Lesson 08 (Mask R-CNN), Phase 4 Lesson 24 (SAM 3)

#Learning Objectives

  • Distinguish tracking-by-detection from query-based tracking and name the algorithm families (SORT, DeepSORT, ByteTrack, BoT-SORT, SAM 2 memory tracker, SAM 3.1 Object Multiplex)
  • Implement IoU + Hungarian assignment from scratch for classic tracking-by-detection
  • Explain SAM 2's memory bank and why it handles occlusion better than IoU-based association
  • Read the three tracking metrics (MOTA, IDF1, HOTA) and pick which one matters for a given use case
Full Access lesson

Keep going. The rest of this lesson is one step away.

Unlock all 503 lessons, video, code, quizzes and every future update. Pay in your own currency with Paystack.

See plans Sign in

The text and code of this lesson are based on AI Engineering from Scratch by Rohit Ghumare, MIT licensed.