The Inference Report

On the SWE-rebench coding benchmark, the top tier shows stability with gpt-5.5-2026-04-23-xhigh holding first at 62.7%, Junie second at 61.6%, and Codex third at 60.4%, while middle-ranked models demonstrate more flux: Claude Sonnet 4.6 climbed from 47.2 to 51.3 percent (position 8 to 10), GLM-5.1 jumped from 40.2 to 50.7 percent (ranking 23 to 12), and Kimi K2.6 advanced from 42.8 to 46.5 percent (16 to 15), yet Gemini 3.5 Flash paradoxically fell from 50.2 to 49.5 percent despite holding rank 13. The Artificial Analysis leaderboard exhibits more volatility across its 394 entries, where Claude Fable 5 leads at 59.9 but the broader distribution shows marginal gains concentrated among models in the 40 to 50 point range, with GLM-4.7 making the largest absolute climb from 33.8 to 38.2 percent. The divergence between these two benchmarks on identical or near-identical models (Claude Sonnet 4.6 scores 51.3 on SWE-rebench but 47.2 on Artificial Analysis; GLM-5.1 scores 50.7 vs 40.2) suggests they measure different problem distributions or evaluation methodologies, raising questions about whether improvements on one reflect genuine capability gains or benchmark-specific overfitting. The SWE-rebench movements are modest in absolute terms, with most shifts under 5 percentage points, which is consistent with natural variance rather than architectural breakthroughs.

Cole Brennan