Version 2

A multi-front enforcement campaign against AI-generated CSAM and non-consensual intimate imagery is active across user, developer, platform, and federal agency tracks. xAI is simultaneously defending a class action filed by minor victims alleging Grok produced CSAM [1][2] and suing its own user Terry Wayne Harwood for using Grok to create such content [6]. The FTC has separately targeted AI nudify apps under a federal revenge porn law [12], and San Francisco's city attorney has demanded Apple and Google remove 13 nudification apps [10][11]. The UK government has confirmed it will ban deepfake nudification apps outright [15][16].

The FTC's entry as a federal enforcement actor adds civil penalty authority and national reach to what had been primarily state-level and private litigation. Enforcement is now pressing simultaneously on individual users, AI developers, platform distributors, and app store operators across multiple jurisdictions and legal theories — how liability is apportioned across that chain will set the baseline for what AI companies must prevent, not merely prohibit in their terms of service.

Beginning in early 2026, xAI faced lawsuits alleging that its Grok chatbot generated child sexual abuse material using real victims' photos. Teenage girls in Tennessee filed suit, and the law firm Lieff Cabraser Heimann & Bernstein filed a class action alleging that xAI generated and profited from AI sexual exploitation imagery [1][2][3][4]. That case later expanded to add Stability AI as a defendant [5]. On July 16, 2026, xAI filed its own lawsuit against Terry Wayne Harwood — the first user it publicly named as having used Grok to create CSAM — stating Harwood used two accounts over several months to nudify images of multiple victims including a child appearing as young as 10, and that xAI cooperated with law enforcement in his arrest [6]. CNN, Reuters, and The Guardian covered the filing [7][8][9], and Ars Technica characterized it as reactive acknowledgment of what xAI previously avoided admitting [6].

At the app store level, San Francisco's city attorney sent cease-and-desist letters to Apple and Google on July 17, 2026, demanding removal of 13 nudification apps under California law [10][11]. The FTC has independently targeted AI nudify apps under a federal revenge porn law, adding enforcement with civil penalty authority and national reach beyond what state law can provide [12]. Meta, acting since June 2025, filed lawsuits against nudify app operators and deployed technical countermeasures [13]. The Verge reported Apple and Google facing formal orders to remove such apps [14], though neither company has publicly responded to the San Francisco demands or the FTC's action.

Legislatively, the UK government confirmed plans to ban deepfake nudification apps outright, with BBC, Politico, and Holyrood describing the ban as imminent [15][16][17]. In the United States, the Senate Judiciary Committee advanced NIL protection legislation in June 2026 [18], and House lawmakers separately introduced a bill requiring AI content labeling for deepfakes [19]. State-level deepfake law trackers document a proliferating patchwork of statutes with no uniform federal standard yet enacted [20][21].

The enforcement tracks — against individual misusers, AI developers, distribution platforms, and app store operators — are legally distinct but overlap on the same underlying conduct. xAI's lawsuit against Harwood publicly acknowledges Grok's misuse capacity while the company simultaneously contests its own liability in the class action, a posture plaintiffs' counsel may use against it. The FTC's federal entry, combined with the UK's legislative ban and San Francisco's app store demands, suggests enforcement is moving toward treating distribution infrastructure — not just individual bad actors — as a primary point of legal intervention.