Cadre

Embedding Models — The 2026 Deep Dive

Word2Vec gave you a vector per word. Modern embedding models give you a vector per passage, cross-lingual, with sparse, dense, and multi-vector views, sized to fit your index. Pick wrong and your RAG retrieves the wrong thing.

Learn 1 hr Python

Prerequisites: Phase 5 · 03 (Word2Vec), Phase 5 · 14 (Information Retrieval)

#The Problem

Your RAG system retrieves the wrong passage 40% of the time. The culprit is rarely the vector database or the prompt. It is the embedding model.

Choosing an embedding in 2026 means picking across five axes:

Dense vs sparse vs multi-vector. One vector per passage, or one per token, or a sparse weighted bag of words.
Language coverage. Monolingual English models still win on English-only tasks. Multilingual models win when corpora are mixed.
Context length. 512 tokens vs 8,192 vs 32,768 — and real effective capacity is often 60-70% of the advertised max.
Dimension budget. 3,072 floats at full precision = 12 KB per vector. At 100M vectors, storage is $1,300/month. Matryoshka truncation cuts this 4×.
Open vs hosted. Open-weight means you control the stack and data. Hosted means you trade control for always-latest.

This lesson names the tradeoffs so you can pick on evidence, not on whatever was popular last quarter.

#The Problem

Keep going. The rest of this lesson is one step away.