AI Engineering/Phase 11/Lesson 08

Fine-Tuning with LoRA & QLoRA

Full fine-tuning a 7B model requires 56GB of VRAM. You don't have that. Neither do most companies. LoRA lets you fine-tune the same model in 6GB by training less than 1% of the parameters. This isn't a compromise -- it matches full fine-tuning quality on most tasks. The entire open-source fine-tuning ecosystem runs on this one trick.

Build 1.3 hr Python

Prerequisites: Phase 10, Lesson 06 (Instruction Tuning / SFT)

#Learning Objectives

Implement LoRA by injecting low-rank adapter matrices (A and B) into a pretrained model's attention layers
Calculate the parameter savings of LoRA vs full fine-tuning: rank r with d_model dimensions trains 2rd parameters instead of d^2
Fine-tune a model using QLoRA (4-bit quantized base + LoRA adapters) to fit within consumer GPU memory
Merge LoRA weights back into the base model for deployment and compare inference speed with and without adapters

Full Access lesson

Keep going. The rest of this lesson is one step away.

Unlock all 503 lessons, video, code, quizzes and every future update. Pay in your own currency with Paystack.

See plans Sign in

The text and code of this lesson are based on AI Engineering from Scratch by Rohit Ghumare, MIT licensed.