AI Labs Defend Against Model Output Distillation: Meta Restricts Claude Code, Anthropic Accuses Alibaba

Two developments have converged around AI model output protection. Anthropic publicly accused Alibaba of running what it calls the largest known distillation attack on Claude, using approximately 25,000 fraudulent accounts to generate 28.8 million exchanges intended to train Alibaba's own models, and disclosed the campaign to the U.S. government.[2][1][3] Separately, internal Meta documents leaked to The Information show Meta has placed strict limits on how engineers in its applied AI division may use Claude Code and OpenAI's Codex, specifically to prevent competitor model outputs from entering Meta's own training pipelines.[6][12][7] Both OpenAI and Anthropic have terms of service explicitly barring use of their model outputs to develop competing models, though legal experts question whether these provisions are enforceable in court.[7][9][8]

These cases define an emerging category of competitive IP dispute unique to AI: a sufficiently capable model can be queried at scale to produce training data that transfers its capabilities to a rival system, potentially without any traditional intellectual property violation. How labs detect, deter, and legally address this practice will shape the economics of frontier model development and the enforceability of AI terms of service.

Anthropic disclosed to the U.S. government that Alibaba orchestrated what the company describes as the largest known distillation attack on Claude.[1] The operation used approximately 25,000 fraudulent accounts to generate 28.8 million exchanges with the model, with the apparent purpose of producing labeled input-output pairs that could train Alibaba's own systems on Claude's behavior.[2][3] Anthropic described the campaign as 'brazenly' and 'illicitly' extracting AI capabilities.[4] Zvi Mowshowitz's commentary confirms Anthropic has formally accused Alibaba of this operation and framed it in the context of broader U.S.-China AI competition.[5] Alibaba has not issued a public response.

Separately, internal documents obtained by The Information show Meta has prepared guidelines restricting how engineers in its applied AI division use Claude Code and OpenAI's Codex.[6] The concern is straightforward: if Meta engineers use competitor AI tools in their development workflows, the outputs of those tools — code suggestions, explanations, design choices — could find their way into Meta's model training pipelines, constituting unintentional distillation.[7] Analyst Rohan Paul characterized the risk as arising from 'even accidental reuse of Claude or Codex answers' and suggested mitigation strategies including 'ingredient tracking' and clean-room rules separating coding-agent outputs from training datasets.[7] The restriction covers Claude Code (Anthropic) and Codex (OpenAI), the same companies whose terms of service explicitly prohibit using their model outputs to develop competing AI models.[7][8]

The legal landscape around these prohibitions is uncertain. Both OpenAI's and Anthropic's terms of service bar use of outputs to train competing models, but legal commentators have noted these clauses may be difficult to enforce in court, particularly against foreign entities.[9][8] A strong case, as Rohan Paul observed, would typically require evidence such as mass scraping, fake accounts, or internal records showing intentional cloning — precisely the kind of evidence Anthropic claims to have in the Alibaba matter.[7] Commentator Prasenjit Sarkar argued the episode shows that 'the bottleneck in AI capability competition just moved from training to policy enforcement.'[10]

Skepticism exists on the technical framing. One commentator (@96Stats, described as being based in China) dismissed BBC coverage of the Anthropic allegation as written by a 'casual journalist,' arguing that 'extraction' as framed in mainstream reporting does not accurately describe what distillation involves technically.[11] This dissent is a minority view in the coverage but reflects a broader debate about whether systematic API querying constitutes IP misappropriation or merely uses a paid service within its technical terms.