Capstone Lesson 39: Instruction Tuning by Supervised Fine-Tuning

A pretrained base model can extend a sequence but cannot follow an instruction. Supervised fine-tuning is the smallest change that fixes this: feed the model paired examples of an instruction and a desired response, and train the body to predict the response tokens. The trick is that you only want the loss to count the response, not the instruction. This lesson builds an Alpaca-style SFT loop with a custom collate function that masks instruction tokens with `ignore_index=-100`, trains on 200 instruction-response pairs, and evaluates on a held-out split using exact-match.

Build 1.5 hr Python (torch, numpy)

Prerequisites: Phase 19 lessons 30-37 (NLP LLM track: tokenizer, embedding table, attention block, transformer body, pre-training loop, checkpointing, generation, perplexity)

#Learning Objectives

Format paired instruction-response data into a single causal sequence with explicit boundary tokens.
Build a collate function that masks instruction tokens so cross-entropy only counts response tokens.
Train a tiny transformer body under the SFT objective and watch the eval metric move.
Implement greedy and temperature-sampled generation that respects the response-start boundary.
Compute held-out exact-match on generated completions.

#Learning Objectives

Keep going. The rest of this lesson is one step away.