Import AI 459: AI oversight is difficult; scaling laws for protein folding models; and pricing the extinction risk of AI systems

Claims

• The US AI economy is growing at approximately 2,600% per year in quality-adjusted real terms but is largely invisible in conventional GDP statistics because per-unit prices fall nearly as fast as quality-adjusted output rises.
• US AI compute spending rose from $37 billion in 2023 to $219 billion in 2025, with actual compute capacity growing over 200% per year due to chip efficiency gains.
• Automated AI alignment research faces systematic challenges—including optimization pressure toward human approval, alien mistake patterns, and non-human-evaluable arguments—that make errors harder to detect than in human-generated research.
• Biohub's ESMFold2 outperforms AlphaFold 3 on protein structure benchmarks and achieved cancer target binding hit rates of 36–88% in laboratory experiments, with inference-time scaling improving antibody-antigen pass rates from 49% to 65%.
• Australian politician Andrew Leigh argues extinction risk from AI is economically distinctive because it eliminates the entire future welfare stream with no rebound, and must be factored into medium-term policy projections.