AI Engineering/Phase 19/Lesson 59

Vision Transformer Encoder

Patches alone do not see. A 12-layer pre-LN transformer with 12 attention heads turns the sequence of patch tokens into a sequence of contextual tokens, with the CLS token pooling whole-image features in its final hidden state. This lesson is the engine room of every modern vision-language model.

Build 1.5 hr Python

Prerequisites: Phase 19 lessons 30-37 (Track B foundations)

#Learning Objectives

Implement a pre-LN transformer block with multi-head self-attention and a feed-forward sub-layer.
Stack 12 blocks with 12 heads to form a ViT-Base encoder.
Wire the patch front end from lesson 58 into the encoder and run a forward pass.
Verify that the CLS token aggregates information from every patch.

Full Access lesson

Keep going. The rest of this lesson is one step away.

Unlock all 503 lessons, video, code, quizzes and every future update. Pay in your own currency with Paystack.

See plans Sign in

The text and code of this lesson are based on AI Engineering from Scratch by Rohit Ghumare, MIT licensed.