Cadre

Audio Evaluation — WER, MOS, UTMOS, MMAU, FAD, and the Open Leaderboards

You cannot ship what you cannot measure. This lesson names the 2026 metrics for every audio task: ASR (WER, CER, RTFx), TTS (MOS, UTMOS, SECS, WER-on-ASR-round-trip), audio-language (MMAU, LongAudioBench), music (FAD, CLAP), and speaker (EER). Plus the leaderboards where you compare.

Learn 1 hr Python

Prerequisites: Phase 6 · 04, 06, 07, 09, 10; Phase 2 · 09 (Model Evaluation)

#The Problem

Every audio task has multiple metrics, each measuring a different axis. Using the wrong metric is how you ship a model that looks great on your dashboard and terribly in production. The 2026 canonical list:

Task	Primary	Secondary
ASR	WER	CER · RTFx · first-token latency
TTS	MOS / UTMOS	SECS · WER-on-ASR-round-trip · CER · TTFA
Voice cloning	SECS (ECAPA cosine)	MOS · CER
Speaker verification	EER	minDCF · FAR / FRR at operating point
Diarization	DER	JER · speaker confusion
Audio classification	top-1 · mAP	macro F1 · per-class recall
Music generation	FAD	CLAP · listening panel MOS
Audio language model	MMAU-Pro	LongAudioBench · AudioCaps FENSE
Streaming S2S	latency P50/P95	WER · MOS

#The Problem

Keep going. The rest of this lesson is one step away.