AI Engineering/Phase 06/Lesson 14

Voice Activity Detection & Turn-Taking — Silero, Cobra, and the Flush Trick

Every voice agent lives or dies on two decisions: is the user speaking now, and are they done? VAD answers the first. Turn-detection (VAD + silence-hangover + semantic endpoint model) answers the second. Get either wrong and your assistant either cuts users off or never shuts up.

Build 45 min Python

Prerequisites: Phase 6 · 11 (Real-Time Audio), Phase 6 · 12 (Voice Assistant)

#The Problem

Three distinct decisions a voice agent makes on every 20 ms chunk:

Is this frame speech? — VAD. Binary, per-frame.
Has the user started a new utterance? — onset detection.
Has the user finished? — end-pointing (turn-end).

The naive answer (energy threshold) fails on any noise — traffic, keyboards, crowd babble. The 2026 answer: Silero VAD (open, deep-learned) + a turn-detection model (semantic endpointing) + a VAD-calibrated silence hangover.

Full Access lesson

Keep going. The rest of this lesson is one step away.

Unlock all 503 lessons, video, code, quizzes and every future update. Pay in your own currency with Paystack.

See plans Sign in

The text and code of this lesson are based on AI Engineering from Scratch by Rohit Ghumare, MIT licensed.