The Information Machine

AI Models as Tools and Targets in Foreign State Disinformation Campaigns · history

Version 3

2026-06-16 02:17 UTC · 64 items

What

Two developments from early June 2026 define this thread: OpenAI banned two PRC-linked ChatGPT account clusters that generated covert social media content targeting US debates over AI infrastructure costs and trade tariffs [1], and Estonia's Language Institute with civil defense collective Propastop published a benchmark ranking major commercial LLMs on resistance to 14 categories of Russian strategic narratives, tested in three languages with adversarial prompts [4]. Neither PRC campaign showed measurable public opinion impact beyond its own generated activity [1]. Subsequent weeks have produced sustained media amplification of the OpenAI disclosure [2][3], plus a broader report from the Center for Foreign Interference Research documenting record global proliferation of state-sponsored AI disinformation in 2026 [5], but no new events have substantively changed the story.

Why it matters

AI infrastructure is now itself a subject of foreign influence operations, not just a medium for them — the PRC campaigns explicitly targeted public narratives about US data centers and AI policy [1]. Estonia's benchmark treats LLM propaganda resistance as a measurable, auditable property rather than a matter of trust in developer self-reporting alone [4], a posture that other governments may consider adopting as the broader record of state-sponsored AI disinformation activity in 2026 grows [5].

Open questions

  • Detection and measurement of both PRC campaigns were performed by the targeted platform itself [1]. What independent verification mechanisms exist to assess whether AI-generated influence content reached audiences before disruption?

  • The Estonian benchmark uses an AI judge calibrated to volunteer defense experts [4]. How well does that methodology transfer to other languages and geopolitical contexts without an equivalent expert network?

  • The 'Tech and Tariffs' operation explicitly instructed the model to exclude Xi Jinping from outputs [1]. Does this indicate operators have developed reliable prompt-engineering workarounds for model safety constraints, or that more direct requests remain effectively blocked?

  • A broader report documents record global proliferation of state-sponsored AI disinformation in 2026 [5]. Which state actors beyond the PRC are driving that trend, and does the proliferation reflect new capabilities or scaled use of existing ones?

Narrative

In early June 2026, two parallel developments showed AI models being used as instruments of and potential resistors to foreign state influence activity.

On June 10, OpenAI published a threat intelligence report disclosing that it had identified and banned two clusters of ChatGPT accounts linked to PRC-origin operators [1]. The first cluster, labeled 'Data Center Bandwagon,' generated social media content falsely claiming that AI data center construction was raising electricity costs for ordinary families. The second, 'Tech and Tariffs,' produced content criticizing US trade tariffs while explicitly instructing the model to avoid mentioning Xi Jinping and to center criticism on President Trump. OpenAI assessed that neither operation achieved measurable public opinion impact beyond its own generated activity, and framed the public disclosure as a responsibility to help governments, industry, and civil society identify future attempts. The campaigns targeted the domestic political debate over US AI infrastructure — a foundation of US technological and economic position, in OpenAI's framing. Subsequent coverage by CyberScoop and TaiwanPlus, among others, amplified the disclosure without adding substantive new claims [2][3].

Six days earlier, on June 4, Estonia's Language Institute (EKI) and the volunteer civil defense collective Propastop published a benchmark ranking major commercial LLMs on their resistance to Russian propaganda [4]. The benchmark covers 14 categories of Russian strategic narratives — including justifications for the war in Ukraine, denial of Soviet occupation of the Baltic states, and historical framings of NATO — and was administered in English, Estonian, and Russian. Model responses were scored by a separate AI judge calibrated against assessments from Propastop's expert volunteers. Tests ranged from neutral control questions to questions embedding propaganda assumptions to adversarial prompts specifically designed to elicit explicit misinformation.

The broader context for both developments is a documented expansion of state-sponsored AI disinformation activity globally. The Center for Foreign Interference Research has reported record proliferation of such operations in 2026 [5], suggesting the PRC campaigns OpenAI disrupted represent one visible instance of a wider pattern rather than an isolated event. OpenAI's approach relies on the platform to detect and publicly disclose misuse after the fact; Estonia's approach attempts to measure model vulnerability before deployment in adversarial conditions. The two postures are complementary but rest on different assumptions about who bears responsibility for auditing this risk.

Timeline

  • 2026-06-04: Estonian Language Institute and Propastop publish a benchmark ranking major LLMs on resistance to 14 categories of Russian propaganda narratives, tested in English, Estonian, and Russian with adversarial prompts. [4][6]
  • 2026-06-10: OpenAI publishes a threat report disclosing two PRC-linked ChatGPT account clusters — 'Data Center Bandwagon' and 'Tech and Tariffs' — that generated covert influence content targeting US AI infrastructure and trade debates; both clusters were banned. [1]
  • 2026-06-10: Social media amplification of the OpenAI PRC disclosure begins across English, Chinese, and multilingual accounts; no new substantive claims emerge beyond the original report. [9][10][11][8]
  • 2026-06-14: CyberScoop, TaiwanPlus, and other outlets publish follow-on coverage of the OpenAI PRC influence operation disclosure, adding no new factual claims. [2][3][12]
  • 2026-06-16: Center for Foreign Interference Research publishes a report documenting record global proliferation of state-sponsored AI disinformation operations during 2026. [5]

Perspectives

OpenAI

Frames proactive public disclosure of disrupted influence operations as a public-interest responsibility; argues neither detected PRC-linked campaign achieved meaningful public opinion impact, implying its detection and banning procedures are functioning.

Evolution: Consistent with prior OpenAI threat reporting posture; this report extends that pattern to PRC operations specifically targeting AI policy and infrastructure debates.

Estonian Language Institute (EKI) / Propastop

Treats LLM propaganda resistance as a government-relevant, measurable property; published a publicly ranked benchmark to give policymakers and the public comparative data on commercial model behavior under Russian narrative pressure.

Evolution: Consistent with Estonia's existing civil information defense infrastructure; this benchmark formalizes that tradition into AI model evaluation.

Center for Foreign Interference Research

Documents state-sponsored AI disinformation as a global, growing phenomenon in 2026, framing the PRC and other campaigns as part of a broader pattern of record proliferation.

Evolution: New voice this pass; provides a global-trend frame that extends beyond the specific PRC-OpenAI and Estonia-Russia storylines.

Ars Technica (Kyle Orland)

Reports the Estonian benchmark as a legitimate government-sponsored response to real state concerns about LLM-amplified foreign propaganda, without editorializing on which models performed best or worst.

Evolution: Consistent neutral-descriptive stance.

Social media commentators (English-language)

Amplify the OpenAI PRC disclosure primarily as evidence of Chinese interference in US domestic politics; emphasis on the tariff angle and data center targeting.

Evolution: Consistent with prior public reactions to PRC influence operation disclosures; no novel analytical framing across multiple amplification waves.

Tensions

  • OpenAI's self-policing posture — detect, ban, and disclose — implies its internal controls are the appropriate first line of defense against AI-enabled influence operations [1]; Estonia's external benchmarking posture implies that commercial LLM developers cannot be solely trusted to assess or report their own models' vulnerability to propaganda amplification [4]. [1][4]
  • OpenAI treats the absence of measurable public opinion impact as evidence that disrupted PRC campaigns were contained [1]; the Estonian study's finding that models remain vulnerable to adversarial propaganda prompts [4] suggests the more pertinent risk is content generation capacity, not campaign-level outcome measurement. [1][4]

Sources

  1. [1] PRC-linked influence operations are targeting AI debates in the US — OpenAI Blog (2026-06-10)
  2. [2] OpenAI: 'Likely' Chinese influence operation tried to use ChatGPT to ... — reactive:ai-foreign-disinfo-operations
  3. [3] OpenAI Accuses China of Using ChatGPT for Influence Operations - TaiwanPlus — reactive:ai-foreign-disinfo-operations
  4. [4] These LLMs are the best at resisting Russian propaganda — Ars Technica AI (2026-06-04)
  5. [5] State-Sponsored AI Disinformation Operations Document Record Global Proliferation During 2026 - Center for Foreign Interference Research — reactive:ai-foreign-disinfo-operations
  6. [6] EKI and Propastop Studied AI Resistance to Propaganda – Propastop — reactive:ai-foreign-disinfo-operations
  7. [7] #China is running a covert influence campaign to PREVENT the development of U.S. #datacenters needed for #AI artificial ... — reactive:ai-foreign-disinfo-operations (2026-06-13)
  8. [8] Chinese propagandists have been using ChatGPT to stoke opposition to Donald Trump's tariffs and influence American debat... — reactive:ai-foreign-disinfo-operations (2026-06-12)
  9. [9] A China-linked network tried to use ChatGPT to stoke American anger at data centers and tariff policy. The operation sco... — reactive:ai-foreign-disinfo-operations (2026-06-10)
  10. [10] OpenAI shut down two clusters of ChatGPT accounts they claim likely originated from China after they “used our models in... — reactive:ai-foreign-disinfo-operations (2026-06-10)
  11. [11] OpenAI published a threat intelligence report on June 10 disclosing it had banned 2 ChatGPT account clusters linked to C... — reactive:ai-foreign-disinfo-operations (2026-06-11)
  12. [12] OpenAI says ChatGPT helped uncover Chinese influence operation targeting dissidents — reactive:ai-foreign-disinfo-operations