The Inference Report

The SWE-rebench rankings show stability at the top with gpt-5.5-2026-04-23-xhigh holding first place at 62.7%, while Codex and Claude Code follow at 60.4% and 59.6% respectively, unchanged from the previous cycle. Movement in the middle tier reveals two distinct patterns: Gemini 3.1 Pro Preview dropped from position 4 to 10 on SWE-rebench, falling from 57.2% to 51.1%, a 6.1-point decline that marks the most substantial regression in the visible rankings. Conversely, GLM-5.1 held ground at 50.7% while rising slightly in Artificial Analysis from 51.4 to maintain position 11, and Kimi K2.6 declined sharply from position 8 at 53.9% to position 13 at 46.5% on SWE-rebench, a 7.4-point drop. GLM-4.7 presents a puzzling divergence: it improved from 38.2% to 42.1% on Artificial Analysis (rising from position 47), yet on SWE-rebench it remained at 38.2% in position 14, suggesting the two benchmarks may measure different problem classes or that the Artificial Analysis score reflects a broader evaluation window. The consistency of scores across both benchmarks for most models in the top 10 indicates reliable measurement, but the divergence for Gemini and Kimi models warrants scrutiny of whether these benchmarks are testing equivalent code-solving difficulty or if recent model updates affected one benchmark more than the other. The lack of movement in the top five positions across both metrics suggests the frontier has stabilized, though the mid-tier churn indicates active differentiation among models in the 45-55% range.

Cole Brennan