Any-Resolution Vision: Patch-n'-Pack and NaFlex

Real images are not 224x224 squares. A receipt is 9:16, a chart is 16:9, a medical scan might be 4096x4096, a mobile screenshot is 9:19.5. The pre-2024 VLM answer — resize everything to a fixed square — threw away the signal that makes OCR, document understanding, and high-resolution scene parsing work. NaViT (Google, 2023) showed you could pack variable-resolution patches into a single transformer batch with block-diagonal masking. Qwen2-VL's M-RoPE (2024) dropped absolute positional tables entirely. LLaVA-NeXT's AnyRes tiled high-resolution images into a base + sub-images. SigLIP 2's NaFlex variant (2025) is now the default encoder for open VLMs that want a single checkpoint to serve every aspect ratio. This lesson implements patch-n'-pack end to end.

Build 2 hr Python (stdlib, patch packer + block-diagonal mask)

Prerequisites: Phase 12 · 01 (ViT patches), Phase 12 · 05 (LLaVA)

#Learning Objectives

Pack patches from a batch of variable-resolution images into one sequence and build the block-diagonal attention mask.
Pick between AnyRes tiling (LLaVA-NeXT), NaFlex (SigLIP 2), and M-RoPE (Qwen2-VL) for a given task.
Compute token budgets for OCR, charts, and photography without resizing.
Name the three failure modes of square-resize: squished text, cropped content, wasted tokens on padding.

#Learning Objectives

Keep going. The rest of this lesson is one step away.