A Red Line and Oversight Framework for Government AI Contracts
Alignment Forum · TurnTrout · 2026-07-18
Former Google DeepMind researcher TurnTrout proposes a detailed corporate governance framework for AI companies contracting with government entities, establishing red lines against autonomous targeting without human control and AI-driven mass profiling, enforced through an internal review body and annual transparency reports.
Extraction
Topics: ai-governanceautonomous-weaponsai-surveillancecorporate-oversightgovernment-ai-contracts
Claims
- A 2026 Fourth Circuit ruling in Al Shimari v. CACI Premier Technology affirmed a $42 million verdict against a defense AI contractor, establishing that AI providers face direct legal liability for harms arising from government contracts even when acting under government direction.
- The framework's Standard 1 prohibits AI use in any system that selects and engages targets for force without appropriate, identifiable human control over each specific engagement decision.
- The framework's Standard 2 prohibits AI from generating individualized profiles, threat assessments, or risk scores from bulk data unless the subject has been individually identified based on particularized facts.
- A seven-person Defense AI Review Body would advise on contract compliance but cannot block contracts; enforcement relies on transparency, with override counts tallied in annual reports visible to all covered AI employees.
- High-risk applications implicating Standards 1 or 2 are restricted to cloud or connected private-cloud deployments where the company retains monitoring and suspension capability; air-gapped deployments are limited to task-specific, capability-scoped models.
Key quotes
The enforcement mechanism is transparency: if the CEO declines to act on a non-compliance finding and the Review Body maintains that finding by majority, override counts are tallied and recorded in the annual transparency report visible to all Covered AI Employees.
The Company may be liable. In Al Shimari v. CACI Premier Technology, Inc. (4th Cir. 2026), the Fourth Circuit affirmed a $42 million jury verdict against a defense contractor for harms arising from services provided under government direction. No court has yet held an AI provider liable on a comparable theory, but The Company does not want to be the test case.
I made the Chief Scientist the single root of trust that everything else hangs off of... The Chief Scientist would staff a Review Body to advise on contracts.