AI Engineering/Phase 04/Lesson 15

Real-Time Vision — Edge Deployment

Edge inference is the discipline of getting a 90-accuracy model to run at 30 fps on a device with 2 GB of RAM. Every percentage point of accuracy is traded against milliseconds of latency.

Learn + Build 1.3 hr Python

Prerequisites: Phase 4 Lesson 04 (Image Classification), Phase 10 Lesson 11 (Quantization)

#Learning Objectives

Measure inference latency, peak memory, and throughput for any PyTorch model, and read the FLOPs / params / latency trade-off
Quantise a vision model to INT8 using PyTorch's post-training quantisation and verify accuracy loss < 1%
Export to ONNX and compile with ONNX Runtime or TensorRT; name the three most common export failures and their fixes
Explain when to pick MobileNetV3, EfficientNet-Lite, ConvNeXt-Tiny, or MobileViT for an edge constraint

Full Access lesson

Keep going. The rest of this lesson is one step away.

Unlock all 503 lessons, video, code, quizzes and every future update. Pay in your own currency with Paystack.

See plans Sign in

The text and code of this lesson are based on AI Engineering from Scratch by Rohit Ghumare, MIT licensed.