Do AI Risks Require Extraordinary Government Intervention?
AI Snake Oil · Sayash Kapoor · 2026-05-21
AI Snake Oil's Sayash Kapoor argues that investing in societal resilience—hardening cybersecurity, biosecurity, and critical infrastructure—is a more durable and less rights-restricting response to AI misuse risks than extraordinary government interventions such as AI nonproliferation regimes.
Extraction
Topics: ai-policyai-safetygovernment-regulationresiliencedual-use-ai
Claims
- Extraordinary government interventions against AI are precautionary, impose costs on companies not directly responsible for misuse, and bypass democratic governance processes.
- AI nonproliferation is far less enforceable than nuclear nonproliferation because there is no physical bottleneck equivalent to enriched uranium and nation-states can match frontier AI capabilities within months.
- Export controls on chips and voluntary predeployment evaluations buy only months of delay before capabilities become widely available through open-weight models.
- Societal resilience—AI-assisted red-teaming, bug bounties, biosecurity screening, and infrastructure hardening—addresses AI misuse without restricting access to beneficial capabilities.
- Governments have underinvested in resilience because it requires polycentric coordination across many agencies, making unilateral executive action more politically tractable despite being less effective.
Key quotes
Nonproliferation is brittle because it relies on a single chokepoint. Resilience distributes defenses across society.
AI is different from nuclear weapons. For one, there is no equivalent 'physical' bottleneck. The core techniques for building AI systems are well known. Adversaries (especially nation-states) can match frontier capabilities within months.
If our main defense against AI risks is nonproliferation, a single technical breakthrough that makes models cheaper to train could be enough to cause instability, especially in a world where we don't also invest in resilience.