Frontier AI Safety Evaluation: Scheming Research and Evaluation Standards
Synthesis history
15 versions, newest first.
-
Version 15 2026-06-20 02:17 UTC · 119 items
No new substantive claims this pass. All new items (31326, 31342, 31343, 31809, 31810, 31811) are tweets and secondary press/blog coverage of the OpenAI Deployment Simulation paper already incorporated in the previous s…
-
Version 14 2026-06-18 18:23 UTC · 113 items
New items this pass are additional coverage of the OpenAI Deployment Simulation paper (PDF, press articles, tweets) already incorporated in the previous synthesis — none contain parseable claims or introduce new substan…
-
Version 13 2026-06-17 08:20 UTC · 105 items
OpenAI published Deployment Simulation (June 16) [^30352][^30345][^30427], a methodology that directly addresses eval-awareness distortion — the most concrete methodological counter to one of the thread's central evalua…
-
Version 12 2026-06-16 02:29 UTC · 97 items
The substantive addition this pass is the no-CoT time horizon research [^29916], which finds GPT-5.5 can complete ~3 minutes of human-equivalent reasoning without chain-of-thought and that no-CoT capability is doubling …
-
Version 11 2026-06-14 18:19 UTC · 82 items
Two substantive findings this pass. Google DeepMind researcher Josh Engels found that SFT-only versions of Gemini 3.1 Pro and Gemini 3 Flash match full production models on safety benchmarks, concluding Gemini's safety …
-
Version 10 2026-06-13 08:18 UTC · 76 items
Two substantive new items this pass. Model diffing agents research [^28093] introduces a complementary evaluation methodology and explicitly names a structural limitation of current frameworks: they can only detect what…
-
Version 9 2026-06-12 02:12 UTC · 70 items
Three substantive new items this pass. Google DeepMind Language Model Interpretability research [^28030] adds a directional finding to the eval-awareness picture: eval-awareness does not uniformly improve behavior — mod…
-
Version 8 2026-06-10 18:13 UTC · 65 items
Three substantive new items this pass. ARC researcher Mikewins published a detailed technical roadmap [^27645] explaining that training-process monitoring is necessary because cryptographic arguments suggest finished mo…
-
Version 7 2026-06-08 02:12 UTC · 57 items
Two substantive new items this pass. Google retroactively asked 404 Media to remove the phrase 'it's critical that we maintain humans in the loop' from a published official statement [^24405], adding a new tension betwe…
-
Version 6 2026-06-04 02:13 UTC · 52 items
ARC's white-box estimation challenge [^23366] is the substantive new development: it introduces a technical argument that black-box behavioral sampling is structurally insufficient to detect control-undermining behavior…
-
Version 5 2026-06-01 18:33 UTC · 45 items
The most significant new development is that OpenAI's playbook reportedly contains a claim that AI capabilities may not be fully evaluable by third parties [^23226], sharpening the structural conflict-of-interest critiq…
-
Version 4 2026-06-01 08:11 UTC · 34 items
Emergence AI's comparative simulation study adds striking empirical evidence that frontier model alignment quality varies dramatically across labs — not just across conditions — with Claude producing zero crimes and Gro…
-
Version 3 2026-05-31 08:10 UTC · 29 items
GovAI enters as a new academic-policy voice with a framework for rigorous third-party frontier AI auditing, reinforcing AVERI's practitioner position with institutional research weight. A technical cryptographic verific…
-
Version 2 2026-05-30 18:44 UTC · 19 items
Three substantive new voices entered this cycle: Anthropic (via 'Teaching Claude Why' and the Feb 2026 Risk Report), AVERI as an independent auditing organization, and EU regulators via AI Act compliance analysis. The c…
-
Version 1 2026-05-30 02:05 UTC · 2 items
Two frontier AI labs published complementary contributions to AI safety evaluation on the same day, May 29, 2026. OpenAI released a methodological playbook intended to standardize how third parties evaluate frontier mod…