Version 7

Four open-weights models released in June–July 2026 — GLM-5.2 (Zhipu AI), Ornith-1.0 (DeepReinforce), VibeThinker-3B (Weibo AI), and Agents-A1 (35B) — have produced benchmark results narrowing capability gaps with closed frontier systems on coding, math, and abstract reasoning[1][5][4][7]. GLM-5.2 holds a verified 22.8% on ARC-AGI-2 at $0.25 per task and leads open-weights models on SWE-Bench, but is under active debate about whether its scores reflect genuine capability or distillation from Claude Opus[3][2]. ByteDance Seed's EdgeBench shifts evaluation from one-shot benchmark performance to in-context experiential learning across 12–72 hour tasks, finding top frontier models roughly doubling their 2-hour learning speed every three months — a dimension most existing benchmarks cannot measure[16]. ARC Prize has announced ARC-AGI-3 milestone prizes[17], indicating the abstract reasoning benchmark landscape is already moving past ARC-AGI-2.

Open-weights models reaching frontier-adjacent scores on established benchmarks offer developers cheaper, inspectable alternatives to proprietary APIs, but the distillation question means those scores may overstate independent capability. EdgeBench's framing is a more fundamental challenge: if what matters for real agentic work is in-context learning speed rather than one-shot task scores, the entire benchmark ranking exercise that animates this thread may be measuring a secondary dimension of the actual gap.

Since mid-June 2026, four open-weights models have produced benchmark results narrowing the gap with leading closed frontier systems. GLM-5.2 from Zhipu AI leads open-weights models on the DeepSWE software engineering leaderboard[1] and scores near Claude Opus 4.7 on traditional LLM benchmarks[2]; the ARC Prize team officially verified its 22.8% on ARC-AGI-2 at $0.25 per task, making it the highest-scoring open-weights model publicly reported on that benchmark, well below GPT-5.5's 85%[3]. VibeThinker-3B from Weibo AI achieves 94.3 on AIME26 and 80.2 Pass@1 on LiveCodeBench v6 using a supervised fine-tuning plus GRPO post-training pipeline, approaching Claude Opus 4.5 from 3 billion parameters under MIT license[4]. Ornith-1.0-397B from DeepReinforce is a Mixture-of-Experts model with 397 billion total parameters and 17 billion active, scoring 82.4 on SWE-Bench Verified under MIT license[5]; Simon Willison tested its 35B GGUF locally and found proficient agentic performance at 103 tokens per second with clean Apache 2.0 licensing[6]. Agents-A1 (35B) takes a different approach, training on 45K-token verified multi-step task trajectories and distilling specialist teacher models for search, science, instruction following, and tool use into a single student model, claiming 1T-parameter-model performance under Apache 2.0[7].

GLM-5.2 has attracted the most analytical attention and the sharpest disagreement. Zvi Mowshowitz initially called it the best available open-weights model but argued it was heavily distilled from Claude Opus — pointing to its tendency to self-identify as Claude and its use of a Claude-based harness — and characterized it as commercially awkward, neither cheap enough for bulk tasks nor strong enough for hard ones[2]. He later framed it as a potential "DeepSeek moment" for open-source agents without resolving the distillation concern[8]. Community evidence points in a more positive direction: a normally benchmark-skeptical observer found GLM-5.2 genuinely impressive in direct use[9], Julian Goldie SEO has reported it beating Claude Opus 4.8 in 4 out of 5 real-world builds[10], developers report real cost savings using it in Cursor[11], and at least one developer confirms DeepSWE benchmark results match hands-on coding experience[12]. A circulating discussion frames GLM-5.2's token volume on DeepSWE as potentially inefficient despite its benchmark wins[13], adding a cost dimension to the commercial viability debate. Separately, an analysis has used GLM-5.2's verified ARC-AGI-2 score combined with WeirdML benchmark results to assemble 8 private benchmark datapoints and estimate closed model capabilities[14][15] — an approach that inherits the distillation caveat: if GLM-5.2's scores partly reflect training on closed-source outputs, those calibration estimates may be systematically off.

A separate methodological development reshapes how to interpret these benchmark comparisons. ByteDance Seed's EdgeBench evaluates AI agents on 134 real-world tasks requiring 12 to 72 hours of interaction, measuring in-context experiential learning rather than one-shot reasoning or static knowledge[16]. After 38,000 total agent-hours of runs, performance as a function of environment interaction time follows a log-sigmoid curve — slow at first, then faster, then leveling off. Top frontier models appear to be roughly doubling their 2-hour learning speed every three months, suggesting a scaling law for environment-based learning. Rohan Paul, who reported the results, frames this as moving evaluation from "what does the model already know?" to "can the model learn while doing real work?" — a distinction that most of the benchmarks used to rank GLM-5.2, Ornith-1.0, and VibeThinker-3B do not capture. ARC Prize has separately announced ARC-AGI-3 milestone prizes[17], indicating the abstract reasoning benchmark is being extended beyond ARC-AGI-2, whose scores have risen from 3% (May 2025) to 85% for GPT-5.5.

The cumulative picture is that open-weights models have made measurable advances on agentic coding, mathematical reasoning, and abstract visual tasks in a compressed period, but gaps to leading closed models remain large on the most demanding benchmarks. The distillation question — whether benchmark scores reflect genuine capability or training on closed-model outputs — is unresolved for both GLM-5.2 and Agents-A1. EdgeBench introduces a distinct concern: if the capability that matters for real agentic work is in-context learning speed rather than one-shot benchmark performance, current rankings may be measuring a secondary dimension of the actual gap.