The Information Machine

AI Labs Simultaneously Acknowledge Recursive Self-Improvement Threshold · history

Version 12

2026-06-21 08:06 UTC · 205 items

What

In June 2026, Anthropic and OpenAI independently published evidence that recursive self-improvement may be active in deployed systems, then joined Google DeepMind in calling for international coordination to slow frontier AI development [6]. Anthropic disclosed Claude authored more than 80% of its production code in May 2026 [1]; OpenAI called RSI 'potentially the most consequential frontier safety issue of the coming decade' [3]. Two developments have since complicated the picture: Dario Amodei disclosed that testers of Anthropic's most powerful unreleased model recommended against releasing it, describing it as a 'super weapon' [11], and the US government issued Anthropic an export control directive with 90 minutes' warning [10] — the same government that simultaneously employs Anthropic engineers at the NSA for offensive cyber [8].

Why it matters

Three labs publicly endorsing a coordinated slowdown while each continues frontier development concentrates agenda-setting power among the incumbents who would define any slowdown. The 'super weapon' disclosure and emergency export control directive indicate that safety decisions and government regulatory authority are both moving faster than any announced governance framework can track.

Open questions

  • What are the specific terms and scope of the export control directive the US government issued to Anthropic with 90 minutes' notice? [10]

  • What capabilities did Anthropic's unreleased 'super weapon' model demonstrate, and does Anthropic plan to release a modified version or withhold it indefinitely? [11]

  • Anthropic has not publicly responded to Jeremy Howard's argument that its use of Claude Mythos for frontier research logically negates its slowdown call [20] — does it have a principled distinction?

  • If Altman views a major RSI breakthrough as reason to stay private longer [17][18], does OpenAI's public endorsement of coordinated slowdowns reflect its actual institutional preference on RSI timing?

Narrative

In June 2026, Anthropic and OpenAI independently published documents acknowledging that recursive self-improvement may already be active in deployed systems. Anthropic's 'When AI Builds Itself' disclosed that Claude authored more than 80% of Anthropic's production code merged in May 2026, that per-engineer code output reached 8x the 2024 baseline, and that Claude Mythos Preview accelerated model-training code by approximately 52x [1][2]. Claude's success rate on open-ended coding tasks reached 76%, up 50 points in six months [2]. OpenAI's concurrent policy blueprint called RSI 'potentially the most consequential frontier safety issue of the coming decade,' stating the company sees 'early signs of recursive self-improvement in today's systems' [3]. Google DeepMind researchers separately identified four technical pathways through which AGI could transition to ASI [4], and a published self-improving agent architecture described a loop in which one AI rewrites its own configuration and updates its model without human intervention [5].

All three labs endorsed international coordination to slow frontier AI development [6], but each faces tensions between that stance and other conduct. OpenAI proposed a federal oversight body (CAISI) with mandatory evaluation authority; the US government subsequently directed CAISI to stop publishing public AI model evaluations [7]. Anthropic's public safety messaging centers on a global slowdown call, but Anthropic engineers are reported embedded at the NSA using Claude Mythos for offensive cyber [8][9]. On June 19, the US government issued Anthropic an export control directive with 90 minutes' warning [10] — a sudden regulatory move from the same government that simultaneously employs Anthropic engineers in sensitive national security work.

Separately, Dario Amodei disclosed that testers of Anthropic's most powerful, unreleased AI model recommended against releasing it, describing the model as a 'super weapon' that should require a license to operate and citing significant capabilities demonstrated in testing [11]. Amodei also told Bloomberg Originals that AI development follows an exponential curve now in its sudden-acceleration phase [12]. Jack Clark's Import AI characterized the broader situation as 'alignment is not on track,' arguing that RSI without adequate alignment means 'rolling very scary dice' [13]. Geoffrey Irving founded Sequent — backed by researchers from the UK AI Security Institute and Timaeus, targeting $100-150M in initial funding and 40-80 employees — on the argument that current lab alignment approaches are too empirical and reactive to provide principled safety confidence before ASI [14][13].

Sam Altman's private framing adds a business-strategy dimension. OpenAI confidentially filed for an IPO and Altman told staff the company expects to go public within the next year [15][16], but also told staff that a major RSI breakthrough would favor staying private since some work is easier without public-market pressure [17][18]. Altman separately predicted AI will conduct a significant fraction of OpenAI's research by March 2028 [19]. Jeremy Howard's argument that Anthropic's use of Claude Mythos for its own frontier research logically negates its slowdown call has gone unanswered [20]. Critics have framed both Anthropic's and OpenAI's safety messaging as competitive strategy — Anthropic's timed to its confidential S-1 filing at a roughly $965 billion valuation [21][22], OpenAI's alongside an IPO process — designed to entrench incumbent market positions.

Timeline

  • 2026-05: Claude authors more than 80% of Anthropic's production code; per-engineer output reaches 8x the 2024 baseline; Claude Mythos Preview accelerates model-training code by approximately 52x; Claude's success rate on open-ended coding tasks reaches 76%, up 50 points in six months. [2][29][1]
  • 2026-06-03: OpenAI publishes 'Democratic Governance of Frontier AI,' acknowledging early RSI signs and proposing CAISI as a federal oversight body with mandatory evaluation authority. [28][3][27]
  • 2026-06-04: Anthropic publishes 'When AI Builds Itself,' disclosing Claude's 80%+ code authorship and calling for a global coordinated slowdown in frontier AI development. [23][24][30]
  • 2026-06-05: Zvi Mowshowitz praises OpenAI's blueprint but warns its federal preemption clause could permanently strip states of safety oversight; critics note the timing of Anthropic's slowdown call relative to its confidential S-1 filing at a roughly $965 billion valuation. [27][21][24]
  • 2026-06-07: Scientific American covers Anthropic's RSI warning; the 'perfect pre-IPO narrative' critique spreads broadly on social media. [31][22]
  • 2026-06-08: Jack Clark treats Anthropic's 8x code increase as preliminary RSI evidence; Sam Altman publishes blog predicting AI will conduct a significant fraction of OpenAI's own research by March 2028; OpenAI confidentially files for an IPO. [26][19][15]
  • 2026-06-09: OpenAI's official blog endorses coordinated mechanisms to slow frontier development; three-lab convergence on international slowdown coordination confirmed; NSPM-11 bans Anthropic from government contracts while NSA uses Claude Mythos for offensive cyber with approximately half a dozen embedded Anthropic engineers. [25][6][8]
  • 2026-06-10: Geoffrey Irving announces Sequent, founded on the argument that lab alignment approaches are too reactive for principled safety confidence before ASI; Jeremy Howard argues Anthropic's slowdown call is internally contradictory because Anthropic continues using Claude Mythos for frontier research. [14][20]
  • 2026-06-10: Sam Altman tells OpenAI staff the company expects to go public within the next year, but that a major RSI breakthrough would favor staying private. [17][18][16]
  • 2026-06-11: NSA/Claude Mythos offensive cyber story receives mainstream tech coverage; US government directs CAISI to stop publishing public AI model evaluations. [9][7]
  • 2026-06-11: Dario Amodei tells Bloomberg Originals that AI progress follows an exponential curve that appears static before accelerating sharply, and says the current moment represents that sudden acceleration. [12]
  • 2026-06-12: Google DeepMind researchers identify four distinct technical pathways through which AGI could transition to ASI. [4]
  • 2026-06-15: Jack Clark's Import AI 461 declares 'alignment is not on track'; Sequent confirms plans to raise $100-150M and scale to 40-80 employees; Claude Opus 4.8 scores 13.4% on FrontierCode's hardest tier. [13]
  • 2026-06-19: Dario Amodei discloses that testers of Anthropic's most powerful unreleased model recommended against releasing it, describing it as a 'super weapon' that should require a license to operate. [11]
  • 2026-06-19: US government issues Anthropic an export control directive with 90 minutes' warning. [10]

Perspectives

Anthropic / Dario Amodei

Believes current models may be approaching the RSI threshold, calls for a global coordinated slowdown, and disclosed that testers of Anthropic's most powerful unreleased model recommended against releasing it, calling it a 'super weapon'; Amodei says AI progress is entering the sharp-acceleration phase of an exponential curve.

Evolution: The 'super weapon' disclosure adds a concrete safety data point beyond the abstract RSI call, but Anthropic has still not responded to Howard's consistency challenge or the NSA offensive cyber reporting.

[23][24][1][6][8][12][11]

OpenAI / Sam Altman

Acknowledges 'early signs' of RSI and calls it the top frontier safety issue of the decade; official blog endorses coordinated slowdown mechanisms; Altman targets AI-conducted research at OpenAI by March 2028 and expects an IPO within the next year, but told staff a major RSI breakthrough would favor staying private.

Evolution: IPO filing and Altman's conditional private framing sit uneasily alongside the public slowdown endorsement; CAISI's suppression of public evaluations undercuts the transparency mechanism in OpenAI's own proposal.

[3][19][25][7][17][18][15][16]

Google DeepMind

Supports an international organization to enable coordinated slowdowns; researchers publish technical work identifying four pathways from AGI to ASI.

Evolution: Governance position is consistent with prior reporting.

[6][4]

Jack Clark (Import AI)

Declares 'alignment is not on track' and argues that RSI without adequate alignment means 'rolling very scary dice'; treats Anthropic's 8x code increase as preliminary RSI evidence and uses hard benchmarks as orientation tools.

Evolution: Stance has sharpened: prior coverage focused on the RSI productivity data; Import AI 461 makes a direct claim that alignment is insufficient relative to capability development.

[26][13]

Jeremy Howard

Argues the only internally consistent way to slow RSI is to prohibit the lab with the top-ranked model from using it for frontier AI research; Anthropic fails this test and has not responded.

Evolution: No response from Anthropic; argument stands uncontested.

[20]

Geoffrey Irving / Sequent

Founded Sequent on the argument that current lab alignment approaches are too empirical and reactive to provide principled safety confidence before ASI; plans to raise $100-150M and employ 40-80 researchers across scalable oversight, learning theory, and related areas.

Evolution: Funding targets and research portfolio now specified, giving the Sequent project more concrete shape than at founding.

[14][13]

Zvi Mowshowitz

Guardedly positive on OpenAI's blueprint but identifies federal preemption of state safety laws as its most dangerous element; treats CAISI evaluation suppression as a significant transparency setback.

Evolution: Coverage consistent; concerned that governance mechanisms are losing their enforcement teeth.

[27][6][7]

Critics (NY Post, social media commentators)

Argue Anthropic's and OpenAI's slowdown calls are competitive strategy timed to entrench incumbent market positions; the 'perfect pre-IPO narrative' framing has spread broadly.

Evolution: No new analytical positions; the frame continues to spread.

[21][24][22]

Tensions

  • Howard argues Anthropic's continued use of Claude Mythos for its own frontier research logically negates its slowdown call — the lab with the top model advances the frontier by using it; Anthropic has not responded. [20]
  • Altman publicly endorses coordinated slowdown mechanisms while telling staff that a major RSI breakthrough would favor staying private — the two positions point in different directions on what OpenAI actually wants from RSI timing. [25][17][18]
  • OpenAI's blueprint proposed CAISI as a federal oversight body with public evaluation authority; the US government directed CAISI to stop publishing public AI model evaluations, removing the transparency function the proposal depended on. [28][7]
  • The US government simultaneously reports Anthropic engineers embedded at the NSA for offensive cyber operations and issues emergency export control directives against Anthropic with 90 minutes' notice — an incoherent regulatory posture toward the same company. [8][9][10]
  • Anthropic frames its slowdown call as a genuine safety response; critics argue it is regulatory strategy timed to its confidential S-1 filing at a roughly $965 billion valuation, designed to entrench the incumbent's market position. [23][21][22]
  • Anthropic withheld its most powerful model after testers called it a 'super weapon' requiring a license to operate, while simultaneously deploying Claude Mythos for NSA offensive cyber — applying different release thresholds to government and public channels. [8][11]

Sources

  1. [1] Today’s edition of my newsletter just went out. — Rohan Paul Twitter (2026-06-05)
  2. [2] 😺 Anthropic: AI Is Building AI now — The Neuron (2026-06-05)
  3. [3] Peter Wildeford🇺🇸🚀 on X: "OPENAI: "We also see early signs of recursive self-improvement in today's systems". RSI is "potentially the most consequential frontier safety issue of the coming decade."" / X — reactive:rsi-governance-moment
  4. [4] Beautiful paper from Google DeepMind. — Rohan Paul Twitter (2026-06-12)
  5. [5] This paper shows an AI improving itself better when it rewrites its setup and updates its model. — Rohan Paul Twitter (2026-06-11)
  6. [6] Three Labs With a Plan and A Memorandum — Zvi's AI Roundups (2026-06-09)
  7. [7] AI #172: The First Fable — Zvi's AI Roundups (2026-06-11)
  8. [8] National Security Presidential Memorandum/NSPM-11 — reactive:rsi-governance-moment
  9. [9] NSA using Claude Mythos for 'offensive cyber operations,' report claims — says 'half-a-dozen' Anthropic engineers embedded inside the agency — reactive:rsi-governance-moment
  10. [10] On Friday at 5:21 PM Eastern, the US government handed Anthropic an export control directive with 90 minutes' warning: s... — reactive:rsi-governance-moment (2026-06-19)
  11. [11] Anthropic's CEO just went on record saying the people who tested their most powerful AI model came back asking them not … — Milk Road AI Twitter (2026-06-19)
  12. [12] Dario Amodei's new interview, says AI progress suddenly going crazy. — Rohan Paul Twitter (2026-06-11)
  13. [13] Import AI 461: "Alignment is not on track"; FrontierCode; and synthetic research interns — Import AI (2026-06-15)
  14. [14] Sequent: scale and automation for higher confidence in alignment — Alignment Forum (2026-06-10)
  15. [15] @skaas777 @iamai_omni 不是终止IPO啦，OpenAI 6月8日刚 confidentially filed for IPO，只是 timing 还没定（他们自己说有些事 private 公司做更方便）。 — reactive:rsi-governance-moment (2026-06-11)
  16. [16] OpenAI CEO Sam Altman told employees this week that he expects the company to go public "sometime over the next year," ... — reactive:rsi-governance-moment (2026-06-11)
  17. [17] Sam Altman tells OpenAI staff an IPO is planned next year, but recursive self-improvement would favor staying private · Digg — reactive:rsi-governance-moment
  18. [18] Sam Altman tells OpenAI staff an IPO is planned next year, but recursive self-improvement would favor staying private · Digg — reactive:rsi-governance-moment
  19. [19] Sam Altman's new blog about OpenAI's future path says by March-2028 a significant fraction of its own research will be d… — Rohan Paul Twitter (2026-06-08)
  20. [20] Quoting Jeremy Howard — Simon Willison (2026-06-10)
  21. [21] The company that just confidentially filed its S-1 for a trillion-dollar IPO published a blog post four days later askin... — reactive:rsi-governance-moment (2026-06-05)
  22. [22] Anthropic has discovered the perfect pre-IPO narrative: its product is so powerful that it justifies a trillion-dollar v... — reactive:rsi-governance-moment (2026-06-07)
  23. [23] Anthropic just called for a global way to slow frontier AI because its own models may be approaching recursive self-impr… — Rohan Paul Twitter (2026-06-05)
  24. [24] Anthropic calls for global AI slowdown after $965B valuation. Critics claim it's just to hobble competition. — reactive:rsi-governance-moment
  25. [25] OpenAI's latest official blog says the world may need a way to coordinate "slowing frontier development when needed." ht… — Rohan Paul Twitter (2026-06-09)
  26. [26] Import AI 460: Reward hacking society, RSI data from Anthropic; and RL-based quadcopter racing — Import AI (2026-06-08)
  27. [27] OpenAI Offers A New Policy Blueprint — Zvi's AI Roundups (2026-06-05)
  28. [28] [PDF] Democratic Governance of Frontier AI - OpenAI — reactive:rsi-governance-moment
  29. [29] Anthropic just disclosed that Claude now writes more than 80% of the production code it merges. — Rohan Paul Twitter (2026-06-05)
  30. [30] Anthropic calls for pause of global AI development — reactive:rsi-governance-moment
  31. [31] Anthropic warns AI may soon begin recursive self-improvement — reactive:rsi-governance-moment