The Information Machine

Frontier AI Safety Evaluation: Scheming Research and Evaluation Standards

Synthesis history

15 versions, newest first.

  1. Version 15 2026-06-20 02:17 UTC · 119 items

    No new substantive claims this pass. All new items (31326, 31342, 31343, 31809, 31810, 31811) are tweets and secondary press/blog coverage of the OpenAI Deployment Simulation paper already incorporated in the previous s…

  2. Version 14 2026-06-18 18:23 UTC · 113 items

    New items this pass are additional coverage of the OpenAI Deployment Simulation paper (PDF, press articles, tweets) already incorporated in the previous synthesis — none contain parseable claims or introduce new substan…

  3. Version 13 2026-06-17 08:20 UTC · 105 items

    OpenAI published Deployment Simulation (June 16) [^30352][^30345][^30427], a methodology that directly addresses eval-awareness distortion — the most concrete methodological counter to one of the thread's central evalua…

  4. Version 12 2026-06-16 02:29 UTC · 97 items

    The substantive addition this pass is the no-CoT time horizon research [^29916], which finds GPT-5.5 can complete ~3 minutes of human-equivalent reasoning without chain-of-thought and that no-CoT capability is doubling …

  5. Version 11 2026-06-14 18:19 UTC · 82 items

    Two substantive findings this pass. Google DeepMind researcher Josh Engels found that SFT-only versions of Gemini 3.1 Pro and Gemini 3 Flash match full production models on safety benchmarks, concluding Gemini's safety …

  6. Version 10 2026-06-13 08:18 UTC · 76 items

    Two substantive new items this pass. Model diffing agents research [^28093] introduces a complementary evaluation methodology and explicitly names a structural limitation of current frameworks: they can only detect what…

  7. Version 9 2026-06-12 02:12 UTC · 70 items

    Three substantive new items this pass. Google DeepMind Language Model Interpretability research [^28030] adds a directional finding to the eval-awareness picture: eval-awareness does not uniformly improve behavior — mod…

  8. Version 8 2026-06-10 18:13 UTC · 65 items

    Three substantive new items this pass. ARC researcher Mikewins published a detailed technical roadmap [^27645] explaining that training-process monitoring is necessary because cryptographic arguments suggest finished mo…

  9. Version 7 2026-06-08 02:12 UTC · 57 items

    Two substantive new items this pass. Google retroactively asked 404 Media to remove the phrase 'it's critical that we maintain humans in the loop' from a published official statement [^24405], adding a new tension betwe…

  10. Version 6 2026-06-04 02:13 UTC · 52 items

    ARC's white-box estimation challenge [^23366] is the substantive new development: it introduces a technical argument that black-box behavioral sampling is structurally insufficient to detect control-undermining behavior…

  11. Version 5 2026-06-01 18:33 UTC · 45 items

    The most significant new development is that OpenAI's playbook reportedly contains a claim that AI capabilities may not be fully evaluable by third parties [^23226], sharpening the structural conflict-of-interest critiq…

  12. Version 4 2026-06-01 08:11 UTC · 34 items

    Emergence AI's comparative simulation study adds striking empirical evidence that frontier model alignment quality varies dramatically across labs — not just across conditions — with Claude producing zero crimes and Gro…

  13. Version 3 2026-05-31 08:10 UTC · 29 items

    GovAI enters as a new academic-policy voice with a framework for rigorous third-party frontier AI auditing, reinforcing AVERI's practitioner position with institutional research weight. A technical cryptographic verific…

  14. Version 2 2026-05-30 18:44 UTC · 19 items

    Three substantive new voices entered this cycle: Anthropic (via 'Teaching Claude Why' and the Feb 2026 Risk Report), AVERI as an independent auditing organization, and EU regulators via AI Act compliance analysis. The c…

  15. Version 1 2026-05-30 02:05 UTC · 2 items

    Two frontier AI labs published complementary contributions to AI safety evaluation on the same day, May 29, 2026. OpenAI released a methodological playbook intended to standardize how third parties evaluate frontier mod…