Capstone Lesson 38: Classifier Fine-Tuning by Head Swap

Track B's first capstone. A pretrained language model is a stack of self-attention blocks ending in a token-prediction head. When you want spam vs ham, the head is wrong but the body is mostly right. This lesson rips the head off, glues a two-class linear layer onto the pooled representation, and trains the classifier two different ways: final-layer only, and full fine-tuning. The eval is precision, recall, and F1 on a held-out split. You learn what each strategy buys you and what it costs.

Build 1.5 hr Python (torch, numpy)

Prerequisites: Phase 19 lessons 30-37 (NLP LLM track: tokenizer, embedding table, attention block, transformer body, pre-training loop, checkpointing, generation, perplexity)

#Learning Objectives

Replace a language-model head with a classification head without re-initialising the body.
Implement two training regimes: frozen body (head-only) and full fine-tuning, sharing one training loop.
Build a tokeniser-aware data pipeline that pads, masks padding, and pools attention output.
Compute precision, recall, F1, and a confusion matrix from raw logits.
Reason about the trade-off between parameter count, training time, and head-room.

#Learning Objectives

Keep going. The rest of this lesson is one step away.