Import AI 457: AI stuxnet; cursed Muon optimizer; and positive alignment
Import AI · Jack Clark · 2026-05-18
Jack Clark's Import AI newsletter covers a pre-Stuxnet cyberweapon (fast16.sys) targeting precision engineering software, a neuron-death flaw in the Muon optimizer with the proposed Aurora fix, a multi-institution position paper on 'positive alignment' for AI flourishing, and Prime Intellect's demonstration of LLMs autonomously improving AI training pipelines.
Appears in
Extraction
Topics: ai-safetyoptimizer-researchcyberweaponspositive-alignmentautonomous-ai-research
Claims
- The fast16.sys virus, predating Stuxnet by years, selectively introduced floating-point errors into precision engineering software used in physics simulations and nuclear-weapons-relevant programs.
- The Muon optimizer causes over 25% of MLP neurons to permanently die during learning-rate warmup, skewing updates toward a small fraction of surviving neurons.
- The Aurora optimizer outperforms Muon on 1.1B-parameter models, improving final loss from 2.31 to 2.26 and boosting MMLU scores by 10 points.
- A multi-institution position paper argues that 'positive alignment'—building AI to actively support human flourishing—is a necessary and distinct complement to safety-focused harm prevention.
- Prime Intellect's agents (Codex running GPT-5.5 and Claude Code with Opus 4.7) beat human baselines on nanoGPT optimization but cannot generate original ideas and rely on upstream human innovations to keep improving.
Key quotes
Why this matters - this is how a superintelligence might prevent others from coming into existence: fast16 is a subtle, hard-to-find bug which has been designed to degrade an actor's ability to do certain types of science.
A model can satisfy all safety constraints while being mediocre, sycophantic, or unhelpful.
The agents tend to add components and rarely run pruning rounds or try removing previous methods. They do not have a good mental model of how components interact.