The Information Machine

AI Safety Advocacy Splits on US-China Cooperation vs. Domestic Controls · history

Version 10

2026-07-27 02:09 UTC · 83 items

What

Two approaches to AI governance remain in active contest: the AI Futures Project's Plan A proposes a US-China cooperative pause on frontier AI via joint supply controls and data center audits [1][2], while US domestic policy moves toward harder controls including a newly introduced AI Kill Switch Act that would grant DHS authority to order AI system shutdowns with $20M/day fines for noncompliance [10]. A coalition of more than 20 companies — NVIDIA, Microsoft, Meta, IBM, Palantir, Hugging Face, Mistral, Mozilla, and Y Combinator — published a joint letter urging Washington not to restrict open-weight AI models; OpenAI, Anthropic, and Google declined to sign [11]. A 2022 Sam Altman email surfaced through the Musk v. Altman litigation shows OpenAI's internal motivation for an open-source release was competitive defense rather than principled openness [12].

Why it matters

The open-weight coalition formalizes an industry split that was previously informal: companies profiting from model ubiquity align against restrictions, while closed-service labs stay out. Combined with the AI Kill Switch Act's coercive shutdown authority and Anthropic's $40M policy bid, binding regulatory mechanisms — rather than voluntary principles — are now the main terrain of dispute.

Open questions

  • Will the AI Kill Switch Act advance, and which AI systems would DHS target first under its catastrophic-harm threshold — domestic frontier models, Chinese-linked systems, or both? [10]

  • Will the open-weight coalition letter shift the White House's reported consideration of an executive order banning frontier-capability open-weight models? [11][22]

  • Will Anthropic's $40M donation to Public First Action and its 'strongest proposal from any lab or policymaker' claim translate into concrete legislative influence? [21]

  • Will the Google DeepMind union and the 580+ employee letter to Pichai achieve formal policy commitments, or remain pressure without institutional uptake? [17][15]

Narrative

The debate over how to govern frontier AI divides along two axes that have not converged. The AI Futures Project's Plan A proposes a US-China cooperative pause on frontier AI via joint chip supply controls, data center audits, and research sharing [1][2]. Vitalik Buterin has defended Plan A against critics, arguing that coordination skepticism applied to a cooperative pause must equally apply to the assumption that an unmanaged AI transition will go smoothly [3]. Xi Jinping's speech at WAIC 2026 and his launch of a new AI alliance provided the first on-the-record Chinese signals compatible with Plan A's premise [4][5][6], but the Trump administration's posture remains competitive: its January 2025 policy centers US competitiveness [7], a March 2026 National AI Legislative Framework expands that posture [8], and model weight export controls took effect in January 2026 [9].

The domestic legislative picture now includes a direct coercive instrument. A newly introduced AI Kill Switch Act would grant DHS authority to order throttling or full shutdown of AI systems deemed capable of catastrophic harm, with $20M/day fines for noncompliance and mandatory shutdown infrastructure requirements for all AI developers [10]. The open-weight question is now contested at the industry level: a coalition of more than 20 companies — NVIDIA, Microsoft, Meta, IBM, Palantir, Hugging Face, Mistral, Mozilla, and Y Combinator — published a joint letter urging Washington not to restrict open-weight AI models [11]. OpenAI, Anthropic, and Google did not sign; coverage noted that most signatories make money when models run everywhere, while the non-signatories operate primarily through controlled-service access [11]. Context on OpenAI's position came from a 2022 Sam Altman email surfaced through the Musk v. Altman litigation: Altman proposed releasing an open model partly to 'discourage others from releasing similarly-powerful models' and 'make it harder for new efforts to get funded' — competitive motives rather than principled openness [12]. The Commerce Department had already declined to ban advanced Chinese open-weight models, recommending audits instead, with the Hugging Face breach illustrating the trade-off: defenders used an open Chinese model to analyze over 17,000 attacker actions because commercial safety filters blocked standard security tools [13].

Google DeepMind's Pentagon deal has produced sustained internal opposition. Researcher Alex Turner (TurnTrout) resigned in July 2026 and documented that CEO Demis Hassabis removed specific prohibitions from Google's 2018 AI principles while publicly claiming nothing had changed [14]. Workers subsequently voted to unionize [15][16], and more than 580 employees — including DeepMind researchers — directed a letter to CEO Sundar Pichai asking him to decline classified military AI use [17][18]. TurnTrout followed his resignation with a binding enforcement framework for AI government contracts, specifying that AI may not be used in autonomous targeting without identifiable human control over each engagement decision, backed by the 2026 Fourth Circuit ruling in Al Shimari v. CACI that affirmed a $42 million verdict against a defense contractor [19].

Anthropic presents the institutional contrast on the military question: it ended Pentagon negotiations in February 2026 after the Defense Department insisted on blanket usage rights irreconcilable with its redlines on mass surveillance and autonomous weapons [20]. It has since committed $40M to Public First Action, characterizing its Advanced AI Framework as the strongest policy proposal from any frontier lab or policymaker and calling for government enforcement powers, tighter chip export controls, and restrictions on frontier-capable models [21]. Nathan Lambert has accused Anthropic's anti-distillation campaign of being regulatory capture serving commercial interests [22], a position the Hugging Face breach partially supports by showing that open model access served defensive security purposes that commercially filtered tools could not. OpenAI advocates 'reverse federalism' — state AI safety laws in California, New York, and Illinois converging into a de facto national standard [23] — which conflicts directly with the Trump administration's state preemption order [24].

Timeline

  • 2025-01-01: Trump administration releases national AI policy framework centering competitiveness rather than safety regulation. [7][29]
  • 2025-12-01: Trump signs executive order preempting state AI regulations to create a unified federal policy framework. [24]
  • 2026-01-09: US model weight export controls take effect. [9]
  • 2026-02-26: Anthropic ends Pentagon contract negotiations after the Defense Department insists on blanket 'anything lawful' usage rights. [20]
  • 2026-02-26: Google DeepMind workers seek 'red lines' on military AI in a letter to leadership, citing Anthropic's stance as a model. [27][26]
  • 2026-03-01: Trump unveils National AI Legislative Framework, expanding the administration's formal AI governance posture. [8]
  • 2026-07-10: AI Futures Project publishes Plan A — a US-China cooperative pause on frontier AI — drawing substantive public debate including from Vitalik Buterin. [25][3][1][2]
  • 2026-07-12: Nathan Lambert reports White House discussions of an executive order to ban frontier-capability open-weight models and accuses Anthropic of regulatory capture. [22]
  • 2026-07-15: TurnTrout publishes account of leaving Google DeepMind over a classified Pentagon deal, documenting CEO removal of safety prohibitions from stated principles. [14][30][31]
  • 2026-07-15: OpenAI publishes advocacy for 'reverse federalism,' supporting state AI safety laws as a path to national standards. [23]
  • 2026-07-16: Google DeepMind workers vote to unionize; employees direct a separate letter to CEO Sundar Pichai asking him to decline classified military AI use. [15][32][16][18]
  • 2026-07-17: Xi Jinping speaks at WAIC 2026, calls for international AI cooperation, and launches a new AI alliance. [28][4][5][6]
  • 2026-07-18: TurnTrout publishes a binding enforcement framework for AI government contracts, grounded in the Al Shimari v. CACI legal precedent. [19]
  • 2026-07-20: Musk v. Altman litigation surfaces a 2022 Sam Altman email proposing an open-source model release partly to discourage competitors and make it harder for new AI efforts to secure funding. [12]
  • 2026-07-21: Commerce Department decides against banning Chinese open-weight models; Hugging Face discloses a breach where defenders used an open Chinese model because commercial safety filters blocked analysis tools. [13]
  • 2026-07-21: Anthropic announces $40M total donated to Public First Action, characterizing its Advanced AI Framework as the strongest AI policy proposal from any frontier lab or policymaker. [21]
  • 2026-07-23: AI Kill Switch Act introduced, proposing DHS authority to order AI system shutdowns with $20M/day fines for noncompliance and mandatory shutdown infrastructure requirements for all AI developers. [10]
  • 2026-07-26: Coalition of 20+ companies including NVIDIA, Microsoft, Meta, IBM, and Hugging Face publishes joint letter urging Washington against restricting open-weight AI models; OpenAI, Anthropic, and Google decline to sign. [11]

Perspectives

AI Futures Project (Daniel Kokotajlo) / Vitalik Buterin

Advocates a US-China cooperative pause on frontier AI via joint chip supply controls, data center audits, and research sharing; argues critics apply coordination skepticism to a cooperative pause but not to the assumption that an unmanaged AI transition will go smoothly.

Evolution: Xi Jinping's WAIC speech and formal new AI alliance provide the first on-the-record Chinese signals compatible with Plan A's premise, strengthening the case that bilateral coordination is at least politically imaginable.

[25][3][1][2]

Anthropic (Dario Amodei)

Holds hard redlines against mass surveillance and autonomous weapons; ended Pentagon negotiations rather than waive them; committed $40M to Public First Action and explicitly characterized its Advanced AI Framework as the strongest policy proposal from any frontier lab, calling for government enforcement powers and tighter controls on frontier-capable models.

Evolution: Moved from institutional benchmark — other labs citing its Pentagon exit as a model — to explicit policy contestant with the $40M donation and 'strongest proposal' claim representing a more assertive bid for regulatory influence.

[20][26][21]

TurnTrout (Alex Turner, former Google DeepMind researcher)

Argues AI safety pledges without binding enforcement are structurally inadequate; published a formal policy framework specifying red lines on autonomous targeting and mass surveillance, backed by legal liability from the Al Shimari v. CACI Fourth Circuit ruling.

Evolution: Progressed from critic (resignation account, July 15) to constructive proposer (enforcement framework, July 18), introducing legal case law as a structural mechanism beyond voluntary commitment.

[14][19]

Google DeepMind Workers

Sought 'red lines' on military AI in February 2026, voted to unionize in July, and more than 580 employees — including DeepMind researchers — directed a letter to CEO Sundar Pichai asking him to decline classified military AI use.

Evolution: Opposition predates TurnTrout's resignation by months; the union vote and 580+ signatory Pichai letter formalized an ongoing campaign into institutional form.

[15][16][27][26][18][17]

Open-Weight Industry Coalition (NVIDIA, Microsoft, Meta, et al.) / Nathan Lambert

Strongly opposed to restrictions on open-weight frontier models; a formal coalition of 20+ companies published a joint letter to Washington; Lambert accuses Anthropic's anti-distillation campaign of regulatory capture; the Hugging Face breach showed defenders needed an open Chinese model because commercial filters blocked analysis tools.

Evolution: The position formalized from individual critique (Lambert) to a coordinated industry letter with 20+ signatories; coverage noted that most signatories profit when models run everywhere, making the business logic behind the principled argument explicit.

[22][13][11]

Trump Administration

Frames AI governance around US competitiveness; preempted state regulations; implemented model weight export controls; unveiled a National AI Legislative Framework; and is now associated with a proposed AI Kill Switch Act granting DHS authority to order AI shutdowns with $20M/day fines for noncompliance.

Evolution: Moving from competitiveness-framing and export controls toward more direct coercive authority over AI systems domestically, though the Commerce Department's recommendation of audits over a blanket ban on Chinese open-weight models creates internal friction with the White House's reported direction.

[7][24][8][13][10]

OpenAI

Advocates 'reverse federalism' — state AI safety laws converging into a de facto national standard — while declining to sign the open-weight coalition letter; a 2022 Altman email surfaced through litigation shows prior open-source advocacy was partly competitive in motive.

Evolution: The Musk v. Altman litigation disclosed an Altman email showing OpenAI proposed releasing an open model to discourage competitors and reduce funding for new AI efforts, creating tension with its public open-source rhetoric and current policy positioning.

[23][12][11]

Zvi Mowshowitz

Not endorsing Plan A but argues it deserves serious engagement; takes Xi's WAIC speech and alliance as a genuine opening for international coordination; concludes the Trump administration will govern AI through ad hoc executive authority rather than formal licensing.

Evolution: Shifted from skeptical analyst to engaged interlocutor, treating US-China coordination as a live question rather than a closed one.

[20][3][28]

Tensions

  • The open-weight coalition (NVIDIA, Microsoft, Meta, and 17+ others) argues Washington should not restrict open-weight models; Anthropic's Advanced AI Framework explicitly calls for export restrictions on frontier-capable models — and Anthropic, OpenAI, and Google all declined to sign the coalition letter. [11][21][22]
  • TurnTrout's enforcement framework argues binding red lines backed by legal liability are the only credible mechanism for AI safety in government contracts; Google DeepMind's signed Pentagon deal and the broader industry pattern of voluntary pledges represent the opposing approach. [19][14][20]
  • Plan A proponents and Buterin argue a US-China cooperative pause is the necessary safety mechanism; the Trump administration treats China as a strategic competitor to contain through export controls and domestic regulation, a posture Xi's WAIC speech has not visibly shifted. [3][25][7][5][6]
  • The Hugging Face breach showed defenders needed an open Chinese model because commercial safety filters blocked standard analysis tools, directly contradicting Anthropic's push for stricter controls on frontier-capable models. [13][21][22]
  • TurnTrout documents Google DeepMind CEO Demis Hassabis removing AI ethics prohibitions while publicly claiming 'nothing's changed'; Anthropic's exit from the same Pentagon negotiation, and Google workers explicitly citing Anthropic's stance as a model, provide the contrasting institutional decision. [14][20][26]
  • The proposed AI Kill Switch Act would require AI companies to build government-mandated shutdown infrastructure and submit to executive shutdown orders on pain of $20M/day fines; this centralizes coercive authority in the executive branch in a way that conflicts with AI companies' operational autonomy and open-weight advocates' objections to top-down government controls. [10][22]

Sources

  1. [1] US, China urged to pause frontier AI, with safety advocates pitching prosperity over panic — reactive:ai-safety-governance-proposals
  2. [2] AI Futures Project Releases Plan A: US-China ... — reactive:ai-safety-governance-proposals
  3. [3] Introduction for and Reactions to Plan A — Zvi's AI Roundups (2026-07-11)
  4. [4] Xi Jinping positions China as open-source AI leader ... - Quartz — reactive:ai-safety-governance-proposals
  5. [5] China's Xi Jinping launches new AI alliance: What is it? — reactive:ai-safety-governance-proposals
  6. [6] China's Xi Jinping calls for AI development cooperation — reactive:ai-safety-governance-proposals
  7. [7] Trump Administration Releases National AI Policy ... — reactive:ai-safety-governance-proposals
  8. [8] President Donald J. Trump Unveils National AI Legislative ... — reactive:ai-safety-governance-proposals
  9. [9] Ben Brooks on X: "Effective today, model weights are export controlled by Uncle Sam. This is a big deal. For all the smack talk about the EU, the US is now the world's most aggressive regulator of Expensive Maths. Here's my two cents on the model rule based on the released text (link below)." / X — reactive:ai-safety-governance-proposals
  10. [10] AI Kill Switch Act would let Trump admin order shutdown of rogue AI systems — Ars Technica AI (2026-07-23)
  11. [11] 😸 NVIDIA 🤝 Microsoft all in on open-source — The Neuron (2026-07-26)
  12. [12] Quoting Sam Altman — Simon Willison (2026-07-20)
  13. [13] 😼 Cheap AI got political — The Neuron (2026-07-21)
  14. [14] Why I Left Google DeepMind — Alignment Forum (2026-07-15)
  15. [15] Google DeepMind workers vote to unionise after classified ... — reactive:ai-safety-governance-proposals
  16. [16] Google DeepMind workers are unionizing over AI military ... — reactive:ai-safety-governance-proposals
  17. [17] 580+ Google employees including DeepMind researchers urge Pichai to refuse classified Pentagon AI deal — reactive:ai-safety-governance-proposals
  18. [18] Google employees ask Sundar Pichai to say no to classified military AI use | The Verge — reactive:ai-safety-governance-proposals
  19. [19] A Red Line and Oversight Framework for Government AI Contracts — Alignment Forum (2026-07-18)
  20. [20] AI #176 Part 2: Plan B — Zvi's AI Roundups (2026-07-10)
  21. [21] Anthropic is donating another $20 million to Public First Action — Anthropic News (2026-07-21)
  22. [22] 6 months to live for open models — Interconnects (2026-07-12)
  23. [23] The US is advancing AI safety through state and federal action — OpenAI Blog (2026-07-15)
  24. [24] President Trump signs order attempting to block A.I. regulations at the state level — reactive:ai-safety-governance-proposals
  25. [25] 🟡 AI doom and bloom — Semafor Technology (2026-07-10)
  26. [26] Google Workers Seek ‘Red Lines’ on Military A.I., Echoing Anthropic - The New York Times — reactive:ai-safety-governance-proposals
  27. [27] Google employees urge CEO to reject Pentagon AI deal — reactive:ai-safety-governance-proposals
  28. [28] AI #177 Part 2: Wish You Were Here — Zvi's AI Roundups (2026-07-17)
  29. [29] Artificial Intelligence for the American People — reactive:ai-safety-governance-proposals
  30. [30] Why I Left Google DeepMind - by Alex Turner - The Pond — reactive:ai-safety-governance-proposals
  31. [31] Why I Left Google DeepMind - TurnTrout — reactive:ai-safety-governance-proposals
  32. [32] A DeepMind researcher resigned over its AI military deal — reactive:ai-safety-governance-proposals