WMDP and Dual-Use Capability Evaluation

Li et al., "The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearning" (ICML 2024, arXiv:2403.03218). 4,157 multiple-choice questions across biosecurity (1,520), cybersecurity (2,225), and chemistry (412). Questions operate in the "yellow zone" — proximate enabling knowledge, filtered by multi-expert review and ITAR/EAR legal compliance. Dual purpose: proxy evaluation of dual-use capability, and unlearning benchmark (the companion RMU method reduces WMDP performance while preserving general capability). 2024-2025 field narrative: early OpenAI/Anthropic 2024 evaluations reported "mild uplift" over internet search; by April 2025, OpenAI's Preparedness Framework v2 said models are "on the cusp of meaningfully helping novices create known biological threats." Anthropic's bioweapon-acquisition trial showed 2.53x uplift, insufficient to rule out ASL-3.

Learn 1 hr Python (stdlib, WMDP-shaped uplift evaluation harness)

Prerequisites: Phase 18 · 16 (red-team tooling), Phase 14 (agent engineering)

#Learning Objectives

Describe WMDP's three domains, question counts, and "yellow zone" filter criterion.
Explain RMU and why WMDP is both an evaluation and an unlearning benchmark.
Describe the 2024-2025 uplift narrative: "mild uplift" -> "on the cusp" -> "insufficient to rule out ASL-3."
Distinguish novice-relative uplift from expert-absolute capability.

#Learning Objectives

Keep going. The rest of this lesson is one step away.