Demis Hassabis on the New Coming Age
Zvi's AI Roundups · Zvi Mowshowitz · 2026-07-19
Zvi Mowshowitz critically reviews Demis Hassabis's AGI governance essay proposing a US Frontier AI Standards Body, then documents Alex Turner's resignation from Google DeepMind after the company signed an unrestricted Pentagon AI contract, breaking its 2018 pledge against autonomous weapons.
Appears in
Extraction
Topics: ai-governanceautonomous-weaponsagi-riskfrontier-ai-policydeepmind
Claims
- Demis Hassabis proposes a US Frontier AI Standards Body modeled on FINRA that would require pre-release model evaluations and could coordinate slowdowns among frontier labs.
- Google DeepMind signed a Pentagon contract with no restrictions on killer robots or mass surveillance, violating the conditions under which DeepMind was originally sold to Google.
- Alex Turner resigned from Google DeepMind after an internal petition of over 250 employees and lobbying of senior leadership including Jeff Dean and Demis Hassabis all failed to stop the Pentagon deal.
- Critics including Nate Soares and Aaron Scher argue Hassabis's proposal is inadequate given that expert disagreement about AI risk spans a range of 5% to 90% probability of catastrophe.
- DeepMind's governance structures have been systematically dismantled since 2014, with the 2018 weapons prohibition removed and the 2026 Pentagon contract representing the culmination of that erosion.
Key quotes
AGI cannot be compared to standard technological breakthroughs, not even ones as consequential as the internet or mobile - it is much more akin to the discovery of electricity or fire. If you stop to think about it, we've essentially found a way to make sand think. It's miraculous. — Demis Hassabis, quoted in the post
I'm glad he acknowledges that 'experts disagree.' I'm annoyed that he glosses past how the disagreement is about whether there's a ~5% or ≥50% chance of total catastrophe. We've gotta do better. — Nate Soares (MIRI), quoted in the post
Many hope that when things get bad enough, someone powerful will say 'no.' I tested that for months. Anthropic defended its red lines, but most did not. Pledges of conscience often vaporize on contact with power. — Alex Turner, quoted in the post