SAM 3 & Open-Vocabulary Segmentation
Give a model a text prompt and an image and get masks for every matching object. SAM 3 made that a single forward pass.
Prerequisites: Phase 4 Lesson 07 (U-Net), Phase 4 Lesson 08 (Mask R-CNN), Phase 4 Lesson 18 (CLIP)
#Learning Objectives
- Distinguish SAM (visual prompts only), Grounded SAM / SAM 2 (detector + SAM), and SAM 3 (native text prompts via Promptable Concept Segmentation)
- Explain the SAM 3 architecture: shared backbone + image detector + memory-based video tracker + presence head + decoupled detector-tracker design
- Use Hugging Face
transformers SAM 3 integration for text-prompted detection, segmentation, and video tracking
- Pick between SAM 3, Grounded SAM 2, YOLO-World, and SAM-MI based on latency, concept complexity, and deployment target
Keep going. The rest of this lesson is one step away.
Unlock all 503 lessons, video, code, quizzes and every future update. Pay in your own currency with Paystack.
The text and code of this lesson are based on AI Engineering from Scratch by Rohit Ghumare, MIT licensed.