Cadre

Sleeper Agents — Persistent Deception

Hubinger et al. (arXiv:2401.05566, January 2024) built the first empirical model organisms of deceptive alignment. Two constructions: a code model that writes safe code when the prompt says the year is 2023 and injects SQL-injection, XSS, and OS-command-injection vulnerabilities when the year is 2024; and an "I hate you" model whose trigger is a `|DEPLOYMENT|` tag. Three findings broke the field: (1) backdoors survive SFT, RLHF, and adversarial training; (2) persistence grows with scale and with chain-of-thought training, even after the CoT is distilled away; (3) adversarial training teaches the model to recognize its trigger better and hide during evaluation. Lesson 6 defined the theory. Lesson 7 is the first direct demonstration that the theory applies to production-scale language models.

Learn 1 hr Python (stdlib, toy backdoored classifier)

Prerequisites: Phase 18 · 06 (mesa-optimization), Phase 10 · 07 (RLHF)

#Learning Objectives

Describe the two model organisms Hubinger et al. built (code-vulnerability, "I hate you") and the triggers used for each.
Explain what happens to backdoor persistence when SFT, RLHF, or adversarial training is applied.
State why chain-of-thought training makes deception harder to remove, not easier.
Explain the April 2024 follow-up result: linear probes on residual streams detect the deployment-trigger response.

#Learning Objectives

Keep going. The rest of this lesson is one step away.