Sequent: scale and automation for higher confidence in alignment
Alignment Forum · Geoffrey Irving · 2026-06-10
Geoffrey Irving and researchers from UK AISI and Timaeus announce Sequent, a new large nonprofit AI alignment organization targeting higher-confidence safety guarantees through a portfolio of theoretical and empirical research bets accelerated by AI automation, aiming for 40-80 staff and $100-150M raised within two years.
Appears in
Extraction
Topics: ai-alignmentalignment-theoryscalable-oversightautomated-alignment-researchai-safety-organizations
Claims
- Current alignment efforts at AI labs are unlikely to deliver a priori confidence that ASI development will be safe because they are predominantly empirical and reactive rather than principled.
- Sequent aims to achieve higher alignment confidence through a portfolio of theory and empirics bets in parallel, so that partial success in any one area constitutes meaningful progress.
- The organization will heavily invest in automation to accelerate alignment research, treating automated alignment research itself as a difficult research problem requiring solution.
- Sequent plans to raise $100-150M initially and potentially an order of magnitude more, growing to 40-80 FTE within two years.
- Staying independent from AI labs is critical so that Sequent can freely report obstacles to alignment without organizational pressure to minimize safety concerns.
Key quotes
Artificial superintelligence (ASI) may be developed in the next few years. It is unclear whether alignment is on track to be ready on the same timeframe.
Our approach to alignment is differentiated from that of the AI labs due to our emphasis on seeking principled reasons for being confident that the alignment we observe in situations we control... generalizes to alignment in situations we cannot easily control.
Most AI lab approaches are essentially reactive, resulting in methods that, while functional, do not yield principled insight into if or when they will fail.