The Information Machine

AI Models as Tools and Targets in Foreign State Disinformation Campaigns · history

Version 2

2026-06-13 18:47 UTC · 55 items

What

Two developments from mid-2026 define this thread: OpenAI banned two PRC-linked ChatGPT account clusters that generated covert social media content targeting US debates over AI infrastructure costs and trade tariffs [1], and Estonia's Language Institute with civil defense collective Propastop published a ranked benchmark assessing how well major commercial LLMs resist 14 categories of Russian strategic narratives, tested across three languages with adversarial prompts [2]. Neither PRC campaign showed measurable public opinion impact beyond its own generated activity [1]. Social media amplification of the OpenAI disclosure has been substantial since June 10, but no new substantive events have emerged.

Why it matters

AI infrastructure is now itself a subject of foreign influence operations, not just a medium for them — the PRC campaigns explicitly targeted public narratives about US data centers and AI policy [1]. Estonia's benchmark treats LLM propaganda resistance as a measurable, auditable property rather than a matter of trust in developer self-reporting alone [2], a posture that other governments may consider adopting.

Open questions

  • Detection and measurement of both PRC campaigns were performed by the targeted platform itself [1]. What independent verification mechanisms exist to assess whether AI-generated influence content reached audiences before disruption?

  • The Estonian benchmark uses an AI judge calibrated to volunteer defense experts [2]. How well does that methodology transfer to other languages and geopolitical contexts without an equivalent expert network?

  • The 'Tech and Tariffs' operation explicitly instructed the model to exclude Xi Jinping from outputs [1]. Does this indicate operators have developed reliable prompt-engineering workarounds for model safety constraints, or that more direct requests remain effectively blocked?

  • Will other governments or international bodies adopt the Estonian model of publicly ranking commercial LLMs on propaganda resistance, or will it remain an outlier approach tied to Estonia's specific civil defense infrastructure [2]?

Narrative

In early June 2026, two parallel developments showed AI models being used as instruments of and potential resistors to foreign state influence activity.

On June 10, OpenAI published a threat intelligence report disclosing that it had identified and banned two clusters of ChatGPT accounts linked to PRC-origin operators [1]. The first cluster, labeled 'Data Center Bandwagon,' generated social media content falsely claiming that AI data center construction was raising electricity costs for ordinary families. The second, 'Tech and Tariffs,' produced content criticizing US trade tariffs while explicitly instructing the model to avoid mentioning Xi Jinping and to center criticism on President Trump. OpenAI assessed that neither operation achieved measurable public opinion impact beyond its own generated activity, and framed the public disclosure as a responsibility to help governments, industry, and civil society identify future attempts. The campaigns were notable for targeting the domestic political debate over US AI infrastructure — a foundation of US technological and economic position, in OpenAI's framing.

Six days earlier, on June 4, Estonia's Language Institute (EKI) and the volunteer civil defense collective Propastop published a benchmark ranking major commercial LLMs on their resistance to Russian propaganda [2]. The benchmark covers 14 categories of Russian strategic narratives — including justifications for the war in Ukraine, denial of Soviet occupation of the Baltic states, and historical framings of NATO — and was administered in English, Estonian, and Russian. Model responses were scored by a separate AI judge calibrated against assessments from Propastop's expert volunteers. Tests ranged from neutral control questions to questions embedding propaganda assumptions to adversarial prompts specifically designed to elicit explicit misinformation.

Taken together, these developments illustrate two structurally distinct responses to the same underlying problem: that large language models can function as content generators for influence operations, and that their default behavior under propaganda-laden prompts varies meaningfully across models and languages. OpenAI's approach relies on the platform to detect and publicly disclose misuse after the fact; Estonia's approach attempts to measure model vulnerability before deployment in adversarial conditions. The two postures are complementary but rest on different assumptions about who bears responsibility for auditing this risk.

Timeline

  • 2026-06-04: Estonian Language Institute and Propastop publish a benchmark ranking major LLMs on resistance to 14 categories of Russian propaganda narratives, tested in English, Estonian, and Russian with adversarial prompts. [2][3]
  • 2026-06-10: OpenAI publishes a threat report disclosing two PRC-linked ChatGPT account clusters — 'Data Center Bandwagon' and 'Tech and Tariffs' — that generated covert influence content targeting US AI infrastructure and trade debates; both clusters were banned. [1]
  • 2026-06-10: Social media amplification of the OpenAI PRC disclosure begins across English, Chinese, and multilingual accounts; no new substantive claims emerge beyond the original report. [6][7][8][5]

Perspectives

OpenAI

Frames proactive public disclosure of disrupted influence operations as a public-interest responsibility; argues neither detected PRC-linked campaign achieved meaningful public opinion impact, implying its detection and banning procedures are functioning.

Evolution: Consistent with prior OpenAI threat reporting posture; this report extends that pattern to PRC operations specifically targeting AI policy and infrastructure debates.

Estonian Language Institute (EKI) / Propastop

Treats LLM propaganda resistance as a government-relevant, measurable property; published a publicly ranked benchmark to give policymakers and the public comparative data on commercial model behavior under Russian narrative pressure.

Evolution: Consistent with Estonia's existing civil information defense infrastructure; this benchmark formalizes that tradition into AI model evaluation.

Ars Technica (Kyle Orland)

Reports the Estonian benchmark as a legitimate government-sponsored response to real state concerns about LLM-amplified foreign propaganda, without editorializing on which models performed best or worst.

Evolution: Consistent neutral-descriptive stance.

Social media commentators (English-language)

Amplify the OpenAI PRC disclosure primarily as evidence of Chinese interference in US domestic politics; emphasis on the tariff angle and data center targeting.

Evolution: Consistent with prior public reactions to PRC influence operation disclosures; no novel analytical framing.

Tensions

  • OpenAI's self-policing posture — detect, ban, and disclose — implies its internal controls are the appropriate first line of defense against AI-enabled influence operations [1]; Estonia's external benchmarking posture implies that commercial LLM developers cannot be solely trusted to assess or report their own models' vulnerability to propaganda amplification [2]. [1][2]
  • OpenAI treats the absence of measurable public opinion impact as evidence that disrupted PRC campaigns were contained [1]; the Estonian study's finding that models remain vulnerable to adversarial propaganda prompts [2] suggests the more pertinent risk is content generation capacity, not campaign-level outcome measurement. [1][2]

Sources

  1. [1] PRC-linked influence operations are targeting AI debates in the US — OpenAI Blog (2026-06-10)
  2. [2] These LLMs are the best at resisting Russian propaganda — Ars Technica AI (2026-06-04)
  3. [3] EKI and Propastop Studied AI Resistance to Propaganda – Propastop — reactive:ai-foreign-disinfo-operations
  4. [4] #China is running a covert influence campaign to PREVENT the development of U.S. #datacenters needed for #AI artificial ... — reactive:ai-foreign-disinfo-operations (2026-06-13)
  5. [5] Chinese propagandists have been using ChatGPT to stoke opposition to Donald Trump's tariffs and influence American debat... — reactive:ai-foreign-disinfo-operations (2026-06-12)
  6. [6] A China-linked network tried to use ChatGPT to stoke American anger at data centers and tariff policy. The operation sco... — reactive:ai-foreign-disinfo-operations (2026-06-10)
  7. [7] OpenAI shut down two clusters of ChatGPT accounts they claim likely originated from China after they “used our models in... — reactive:ai-foreign-disinfo-operations (2026-06-10)
  8. [8] OpenAI published a threat intelligence report on June 10 disclosing it had banned 2 ChatGPT account clusters linked to C... — reactive:ai-foreign-disinfo-operations (2026-06-11)