The Information Machine

AI Safety Advocacy Splits on US-China Cooperation vs. Domestic Controls

open · v1 · 2026-07-10 · 16 items

What

The AI safety community is divided between two governance strategies: the AI Futures Project's 'Plan A,' which calls for a temporary US-China cooperative pause on frontier AI development while existing capabilities drive economic growth [1], and a domestic-controls approach centered on export restrictions and executive authority [3][2]. The Trump administration has declined to build a formal AI licensing regime, leaving governance to ad hoc executive discretion [4]. Anthropic's Pentagon partnership collapsed after the Defense Department demanded blanket usage rights that overrode Anthropic's stated limits on mass surveillance and autonomous weapons [4].

Why it matters

Whether AI safety is treated as a cooperative global problem or a unilateral national security matter has concrete consequences: a cooperative pause requires Chinese participation that is politically contested, while purely domestic export controls risk pushing capability development into less safety-conscious hands. The Anthropic-Pentagon breakdown shows these are not abstract debates—redlines either hold or they don't when a major contract is on the table.

Open questions

  • Can the AI Futures Project's 'Plan A' secure meaningful Chinese participation in a cooperative pause given current US-China strategic competition, and what verification mechanism could make it credible? [1]

  • Will the Trump administration's reliance on ad hoc executive authority over AI produce coherent governance, or the opaque, arbitrary regime critics predict? [4]

  • Do antitrust laws prevent frontier AI labs from lawfully agreeing to a coordinated pause, and what regulatory authorization could enable one? [5][7]

  • Now that model weights face export controls, how will that regime apply to open-weight releases approaching the capability threshold described as 'somewhere between Opus 4.8 and Mythos'? [4][3]

Narrative

Two governance visions are competing within the AI safety community. The AI Futures Project's 'Plan A' argues that the US and China should agree to a temporary pause on frontier AI development while existing capabilities—already sufficient to drive substantial economic growth given adequate data-center build-out—are deployed and safety infrastructure is constructed [1]. Proponents like Daniel Kokotajlo frame this not as anti-growth but as a prerequisite for capturing AI's benefits: 'If you want to actually benefit from that growth and abundance, you need to avoid the risks involved' [1]. The cooperative approach faces obvious practical obstacles, including the question of verification and what one account describes as 'pretty far-out ways of ensuring China and other countries don't cheat to get ahead' [1].

The competing approach, largely reflected in current US policy, emphasizes domestic controls and competitive advantage. The Trump administration issued an executive order preempting state AI regulations in December 2025 [2] and moved toward export controls on model weights [3], which one observer characterized as making the US 'the world's most aggressive regulator of Expensive Maths' [3]. Analyst Zvi Mowshowitz argues the administration will not build a formal licensing regime, leaving governance dependent on opaque executive discretion—'winners chosen by the whims of those in power' [4]. Open-weight models at roughly 'Mythos-level' capability are now treated by both US and Chinese governments as posing credible national security risks, a threshold shaping both export controls and voluntary release decisions [4].

The Anthropic-Pentagon episode illustrates where these governance visions collide institutionally. Anthropic ended contract negotiations with the Department of War after the Pentagon insisted on blanket 'anything lawful' usage rights. Dario Amodei stated in February 2026 that this posture eliminates Anthropic's redlines on mass surveillance and autonomous weapons targeting, leaving 'no way forward' [4]. Separately, Anthropic and AE Studio developed GRAM (gradient-routed auxiliary module), a technique that routes dual-use capabilities into removable model compartments rather than suppressing them through safety training—a technical architecture that could allow selective capability unlocking depending on who controls the module [4].

Legal and governance analysts have examined whether labs can even coordinate on a pause. A Lawfare analysis argues a coordinated pause may be the most valuable safety intervention available, but antitrust constraints complicate direct lab-to-lab agreements without regulatory authorization [5][6]. The Centre for the Governance of AI has proposed an evaluation-based coordination scheme as a potentially workable legal path [7]. The Trump administration's national AI policy framework frames competitiveness rather than safety as the organizing principle [8][9], while Brookings has analyzed the broader structural competition between US and Chinese AI strategies [10].

Timeline

  • 2025-01-01: Trump administration releases national AI policy framework centering competitiveness rather than safety regulation as the governing principle. [8][9]
  • 2025-12-01: Trump signs executive order preempting state AI regulations to create a unified federal policy framework. [2][11]
  • 2026-01-09: US model weight export controls take effect, prompting commentary that the US has become the world's most aggressive AI regulator. [3]
  • 2026-02-26: Anthropic ends Pentagon contract negotiations after the Defense Department insists on blanket 'anything lawful' usage rights, citing irreconcilable conflicts with Anthropic's redlines on mass surveillance and autonomous weapons. [4]
  • 2026-07-10: AI Futures Project's 'Plan A'—a US-China cooperative pause on frontier AI—receives media attention as a safety-compatible growth alternative to competitive domestic controls. [1]
  • 2026-07-10: Zvi Mowshowitz publishes 'Plan B' analysis concluding the Trump administration will govern AI through ad hoc executive authority rather than formal licensing, and that open-weight advocacy at Mythos-level capability ignores established national security risks. [4]

Perspectives

AI Futures Project (Daniel Kokotajlo)

Advocates a temporary US-China cooperative pause on frontier AI while existing capabilities drive economic growth and safety infrastructure is built; frames safety and growth as compatible rather than competing goals.

Evolution: Newly prominent voice representing a cooperative framing distinct from both purely adversarial US-China competition and unilateral domestic-controls approaches.

Anthropic (Dario Amodei)

Holds hard redlines against mass surveillance and autonomous weapons targeting as conditions for government partnerships; ended Pentagon negotiations rather than waive them. Pursuing technical safety work including GRAM architecture for capability compartmentalization.

Evolution: The February 2026 Pentagon contract collapse represents a concrete test of stated limits; Anthropic declined a major government contract rather than compromise its redlines.

Zvi Mowshowitz

Skeptical of ad hoc Trump governance; alarmed that capability growth has reached a threshold where open-weight models pose genuine national security risks; critical of open-weight advocates who discount those risks.

Evolution: Consistent skeptical-analytical stance; 'Plan B' framing reflects pessimism that formal governance structures will emerge from the current administration.

Trump Administration

Frames AI governance around US competitiveness; preempted state regulations; implemented model weight export controls; declined to build a formal licensing regime in favor of executive discretion.

Evolution: Policy has moved from early pro-growth deregulatory posture toward selective national-security-based export controls while resisting domestic regulation or formal licensing.

Open-Weight Advocates

Continue to push for public release of frontier-capability model weights, arguing openness enables innovation and resists AI power centralization.

Evolution: Facing increasing pressure as capability thresholds shift; Zvi argues their position amounts to a 'rock with Open Is Good written on it' that ignores established security risks at Mythos-level capability.

Legal and Governance Analysts (Lawfare, Centre for the Governance of AI)

Coordinated pausing by frontier labs may be legally achievable through evaluation-based coordination schemes, and may be the most valuable safety intervention available, though direct lab-to-lab agreements face antitrust constraints.

Evolution: Emerging body of analysis on the legal mechanics of cooperative pausing; the Lawfare piece is explicitly supportive of the coordinated pause as a policy goal.

Semafor Technology

Reportorially neutral, editorially favorable to Plan A's framing that safety and economic growth need not conflict; suggests this pitch may be more politically viable than prior doom-focused safety advocacy.

Evolution: Consistent reportorial voice; 'doom and bloom' framing reflects an observation that AI safety messaging itself is evolving toward growth compatibility.

Tensions

  • The AI Futures Project argues a US-China cooperative pause is the right safety mechanism; the Trump administration's policy treats China as a strategic competitor to contain through export controls, not a partner in cooperative governance. [1][2][8]
  • Anthropic holds that mass surveillance and autonomous weapons targeting are outside the scope of any government contract; the Pentagon argues its posture requires blanket 'anything lawful' usage rights with no carve-outs. [4]
  • Open-weight advocates argue frontier model weights should be publicly released; the US government now treats Mythos-level open weights as a credible national security risk subject to export controls. [4][3]
  • Critics argue formal licensing is needed for coherent AI governance; the Trump administration has explicitly declined to build one, preferring ad hoc executive authority. [4][8]
  • Legal analysts argue a coordinated lab pause could be the most valuable safety intervention available; antitrust law may prevent labs from agreeing to one without regulatory authorization. [7][5]

Status: active and growing

Sources

  1. [1] 🟡 AI doom and bloom — Semafor Technology (2026-07-10)
  2. [2] Ensuring a National Policy Framework for Artificial Intelligence — reactive:us-ai-policy-regulation
  3. [3] Ben Brooks on X: "Effective today, model weights are export controlled by Uncle Sam. This is a big deal. For all the smack talk about the EU, the US is now the world's most aggressive regulator of Expensive Maths. Here's my two cents on the model rule based on the released text (link below)." / X — reactive:ai-safety-governance-proposals
  4. [4] AI #176 Part 2: Plan B — Zvi's AI Roundups (2026-07-10)
  5. [5] Can Frontier AI Labs Lawfully Agree to Pause? | Lawfare — reactive:ai-safety-governance-proposals
  6. [6] Lawfare - A coordinated pause by frontier AI labs may be... — reactive:ai-safety-governance-proposals
  7. [7] An evaluation-based coordination scheme for frontier AI ... — reactive:ai-safety-governance-proposals
  8. [8] Trump Administration Releases National AI Policy ... — reactive:ai-safety-governance-proposals
  9. [9] Artificial Intelligence for the American People — reactive:ai-safety-governance-proposals
  10. [10] Competing AI strategies for the US and China | Brookings — reactive:us-ai-policy-regulation
  11. [11] President Trump signs order attempting to block A.I. regulations at the state level — reactive:ai-safety-governance-proposals