The Inference Report

The SWE-rebench rankings show stability at the top tier, where gpt-5.5-2026-04-23-xhigh holds 62.7%, Junie 61.6%, and Codex 60.4%, with no movement in the first nine positions. Below that band, modest reshuffling reflects incremental gains across mid-tier models. Claude Sonnet 4.6 climbed from position 10 to maintain its 51.3% score, while GLM-5.1 advanced from rank 23 to 12 by improving from 40.2% to 50.7%, a 10.5-point jump that signals either a methodology change, model update, or evaluation refinement worth scrutinizing. Gemini 3.5 Flash dropped from 7 to 13 despite holding 49.5%, suggesting the ranking absorbed new entrants or recalibration. The Artificial Analysis benchmark, by contrast, saw more substantial motion: Grok Build 0.1 0616 entered at rank 28, while Ring-1T appeared at 159 without prior placement, indicating either fresh model releases or expanded coverage. At the lower end, the data reveals compression around single-digit scores, where models cluster densely and small score shifts produce large rank swings, making those positions less meaningful as discriminators. The movement pattern suggests the SWE-rebench is maturing into a stable ordering of proven performers, while Artificial Analysis continues absorbing new competitors, though neither benchmark's methodology is transparent enough to confirm whether score changes reflect genuine capability shifts or evaluation adjustments.

Cole Brennan