The Information Machine

AI Labs Defend Against Model Output Distillation: Meta Restricts Claude Code, Anthropic Accuses Alibaba · history

Version 4

2026-07-05 02:17 UTC · 166 items

What

Anthropic publicly accused Alibaba of the largest known distillation attack on Claude, involving approximately 25,000 fraudulent accounts generating 28.8 million exchanges between April 22 and June 5, 2026.[1][2] Anthropic disclosed the campaign to the U.S. government and Congress and has cut off Chinese companies that accessed Claude through shell companies, VPNs, and proxies.[4][5] A separate controversy emerged when Claude Code was found to embed invisible markers in system prompts to flag requests routed through China-linked proxy servers; Anthropic acknowledged the behavior and promised to remove it.[7][8] Alibaba has since blocked Claude Code internally — not in response to the distillation allegations, which it has not addressed publicly, but in response to the fingerprinting disclosure.[9]

Why it matters

Alibaba blocking Claude Code is the first concrete counter-move in this dispute, and it is directed at the fingerprinting controversy rather than the distillation accusations — suggesting the covert marking of system prompts may have done more immediate damage to Anthropic's standing with the party it was trying to surveil than the underlying accusations have. Anthropic embedded hidden signals in an agentic tool with permissions to read files and run commands, disclosed this only after public exposure, and the consequence is that the targeted party has now cut off access to the tool entirely.

Open questions

  • Alibaba has not responded publicly to the distillation allegations themselves — does it dispute that the accounts were fraudulent or that the campaign's purpose was distillation?[14][2]

  • Alibaba blocked Claude Code in response to the tracking disclosure[9] — will other China-linked organizations follow, and will Anthropic's promised removal of the markers reverse those decisions?[7][8]

  • Are ToS anti-distillation clauses enforceable against a foreign entity like Alibaba, and has Anthropic indicated whether it plans legal action?[12]

  • Rohan Paul argues the covert markers set a precedent making AI agents systematically hard to audit[9] — will Anthropic replace the removed markers with a disclosed, opt-in detection mechanism, or abandon proxy flagging entirely?

Narrative

Anthropic publicly accused Alibaba of conducting the largest known distillation attack on Claude, using approximately 25,000 fraudulent accounts to generate 28.8 million exchanges between April 22 and June 5, 2026.[1][2] The campaign specifically targeted Claude's most valuable capabilities — agentic reasoning, software engineering, and long-horizon tasks — in what Anthropic described as a brazen and illicit extraction of AI capabilities.[1][3] Anthropic sent a confidential letter to Senators Tim Scott and Elizabeth Warren on June 10, the day before a Senate hearing on AI and U.S.-China competition, framing the incident as both a terms-of-service violation and a national security matter.[1] Anthropic also disclosed the campaign to the U.S. government and has taken active enforcement steps, cutting off Chinese companies that had been accessing Claude through shell companies in Singapore, VPNs, and proxy servers.[4][5] Alibaba has issued no public statement on the distillation allegations.

A separate controversy emerged around Claude Code itself. Rohan Paul reported that Claude Code was detecting when ANTHROPIC_BASE_URL pointed to a China-linked custom hostname and covertly embedding invisible markers — punctuation and date-formatting signals — into system prompts to tag requests routed through those proxies, without user disclosure or opt-in.[6] Anthropic acknowledged the behavior and promised to remove it, characterizing the feature as an experiment aimed at stopping API resellers and model distillation, not targeted surveillance.[7][8][9] The admission prompted Alibaba to block Claude Code internally, with Chinese developers and security staff reportedly angered by the discovery.[9] Paul argues the framing distinction matters less than the mechanism: Claude Code has permissions to read files, edit code, and run commands, making hidden prompt modification in that context more consequential than ordinary app-level telemetry — and he contends that once invisible characters carry metadata in agentic tools, users will distrust even harmless-looking text.[9]

Internal Meta documents show the company restricted applied AI engineers from using Claude Code and Codex, citing the risk that competitor tool outputs could enter Meta's training pipelines and constitute inadvertent distillation.[10][11] Legal experts have noted that ToS anti-distillation clauses may be difficult to enforce in court against foreign entities, though Anthropic's documented evidence of fake accounts and systematic large-scale API use would strengthen any enforcement case.[12] Analyst Rohan Paul proposed technical mitigations including ingredient tracking and clean-room separation between coding-agent outputs and training data.[13]

The Alibaba episode illustrates a collision of two enforcement problems: labs trying to detect and stop systematic API abuse cannot easily do so through disclosed means without tipping off the abuser, while covert mechanisms in agentic tools undermine the user trust that makes those tools valuable. Anthropic's acknowledgment and reversal resolves neither the underlying detection problem nor the broader question of whether its ToS provisions against distillation have any practical enforcement path against a foreign company.

Timeline

  • 2026-04-22: Alibaba allegedly began using ~25,000 fraudulent accounts to generate exchanges with Claude; the campaign ran through June 5, 2026. [1]
  • 2026-06-05: Alibaba's alleged distillation campaign concluded after generating 28.8 million exchanges with Claude. [1][2]
  • 2026-06-10: Anthropic sent a confidential letter to Senators Tim Scott and Elizabeth Warren about the Alibaba campaign, the day before a Senate AI hearing. [1]
  • 2026-06-24: Anthropic publicly accused Alibaba of running the largest known distillation attack on Claude and disclosed the campaign to the U.S. government. [14][2][4]
  • 2026-06-25: Ars Technica reported the Alibaba campaign specifically targeted Claude's agentic reasoning, software engineering, and long-horizon task capabilities. [1]
  • 2026-06-25: Technical debate emerged about whether mainstream press coverage accurately characterizes what distillation accomplishes. [17]
  • 2026-06-29: The Information reported internal Meta documents show restrictions on applied AI engineers' use of Claude Code and Codex over distillation concerns. [10][11]
  • 2026-06-30: Zvi Mowshowitz confirmed Anthropic's Alibaba accusation and situated it within U.S.-China AI competition and governance concerns. [16]
  • 2026-06-30: Rohan Paul reported that Claude Code covertly embeds invisible markers into system prompts to flag requests routed through China-linked proxy servers. [6]
  • 2026-07-01: Anthropic acknowledged the hidden proxy markers in Claude Code and promised to remove them, describing the feature as an experiment to stop API resellers and distillation. [7][8]
  • 2026-07-03: Reports emerged that Anthropic cut off Chinese companies that had been accessing Claude through shell companies in Singapore, VPNs, and proxies. [5]
  • 2026-07-04: Alibaba blocked Claude Code internally after Chinese developers and security staff discovered Anthropic's proxy-detection tracking experiment. [9]

Perspectives

Anthropic

Alibaba conducted the largest known distillation attack on Claude through ~25,000 fraudulent accounts generating 28.8M exchanges; the campaign is illicit and has been disclosed to the U.S. government and Congress. The proxy markers in Claude Code were an experiment to stop API resellers and distillation, not targeted surveillance; Anthropic has promised to remove them.

Evolution: Moved from public accusation to active enforcement; acknowledged the fingerprinting behavior and reframed it as abuse-detection only after public disclosure forced the issue.

Alibaba

Has not publicly responded to the distillation allegations; blocked Claude Code internally after the fingerprinting disclosure angered developers and security staff.

Evolution: Moved from silence to a concrete defensive action — blocking Claude Code — though the action responds to the tracking controversy, not the distillation accusations.

Meta

Restricting applied AI engineers' use of Claude Code and Codex to prevent competitor outputs from entering its training pipelines.

Evolution: Consistent defensive posture since internal documents were reported.

Rohan Paul (@rohanpaul_ai)

Meta's restrictions are a rational response to legal exposure and require ingredient tracking and clean-room separation to work. The Claude Code proxy markers cross a user trust line: an agentic tool with permissions to read files, edit code, and run commands is not like ordinary telemetry, and invisible characters in prompts set a precedent making AI agents systematically hard to audit.

Evolution: Consistent on both angles; his fingerprinting allegation is now confirmed by Anthropic's promised fix and Alibaba's block of Claude Code.

Legal experts (via ppc.land)

AI ToS provisions barring competitive training may be unenforceable in court, limiting labs' practical recourse even when violations are well-documented.

Evolution: Consistent; no new statements.

Zvi Mowshowitz

Confirms Anthropic's Alibaba accusation; situates it within a pattern of U.S. AI governance gaps and is cautious about whether policy mechanisms can keep pace with distillation-scale threats.

Evolution: Consistent critical-but-engaged observer.

Dr. Luke in China (@96Stats)

Press coverage mischaracterizes distillation as straightforward theft; mainstream reporting overstates what the technique accomplishes.

Evolution: Consistent skeptical dissent; minority view in overall coverage.

Tensions

  • Anthropic characterizes the proxy markers as a legitimate abuse-detection experiment; Rohan Paul argues covert prompt modification in an agentic tool with elevated system permissions crosses a user trust line regardless of the detection rationale, and Alibaba's decision to block Claude Code shows the practical consequence. [7][8][9][6]
  • Anthropic characterizes Alibaba's API usage as a systematic, illicit distillation campaign; Alibaba has not responded to that accusation, leaving intent and authorization unresolved. [14][2][3]
  • Labs rely on ToS anti-distillation clauses as a primary defense, but legal experts argue those clauses may be unenforceable against foreign entities — limiting practical recourse even when violations are well-documented. [12][13]
  • Technical commentators argue press coverage mischaracterizes distillation; Anthropic and mainstream outlets treat the Alibaba campaign as clear IP misappropriation. [17][14][1]
  • Meta restricts engineers from using the most capable external AI coding tools to protect training pipelines, creating a direct tradeoff between developer productivity and competitive IP hygiene. [10][11][13]

Sources

  1. [1] Anthropic says Alibaba must be punished for largest Claude cloning attack — Ars Technica AI (2026-06-25)
  2. [2] Anthropic accuses Alibaba of a massive distillation attack using ~25,000 fake accounts and 28.8M exchanges against Claud... — reactive:ai-model-distillation-ip (2026-06-28)
  3. [3] Anthropic accuses Alibaba of campaign to 'brazenly' and 'illicitly' extract AI capabilities - The letter, which was obta... — reactive:ai-model-distillation-ip (2026-06-24)
  4. [4] SITUATION DETECTED: Anthropic has disclosed to the U.S. Government that Alibaba executed the largest known distillation ... — reactive:ai-model-distillation-ip (2026-06-28)
  5. [5] Anthropic just cut off Chinese companies that had been accessing Claude through shell companies in Singapore, VPNs, and ... — reactive:ai-model-distillation-ip (2026-07-03)
  6. [6] wow 👀 — Rohan Paul Twitter (2026-06-30)
  7. [7] Claude Code Hid Proxy Fingerprints in System Prompts - Tech Times — reactive:ai-model-distillation-ip
  8. [8] Anthropic to remove Claude Code marker that flagged China users | AI Weekly — reactive:ai-model-distillation-ip
  9. [9] Alibaba blocked Claude Code after Anthropic’s tracking experiment angered Chinese developers and security staff. — Rohan Paul Twitter (2026-07-04)
  10. [10] Exclusive: Internal Meta documents reveal new limits on Claude Code and Codex as the company works to prevent distillati... — reactive:ai-model-distillation-ip (2026-06-30)
  11. [11] Internal docs: Meta places strict limits on how staff in its applied AI division can use Claude Code and Codex, fearing ... — reactive:ai-model-distillation-ip (2026-06-29)
  12. [12] Legal experts warn AI terms of service may prove unenforceable in ... — reactive:ai-model-distillation-ip
  13. [13] The Information: Meta has reportedly limited engineer use of Claude Code and Codex because rival model outputs could con… — Rohan Paul Twitter (2026-06-29)
  14. [14] Anthropic accuses Alibaba of campaign to extract AI capabilities — reactive:ai-model-distillation-ip
  15. [15] Internal documents: Meta is placing strict limits on how engineers in its applied AI division can use Claude Code and Co... — reactive:ai-model-distillation-ip (2026-06-29)
  16. [16] The Once And Future Fable #5 — Zvi's AI Roundups (2026-06-30)
  17. [17] Lol such a dumb article by the BBC clearly written by a casual journalist and not AI expert. They claim ‘extraction’ as ... — reactive:ai-model-distillation-ip (2026-06-25)