The Inference Report

The SWE-rebench rankings remained static across the top tier, with gpt-5.5-2026-04-23-xhigh holding first place at 62.7% and the next five positions unchanged. The Artificial Analysis benchmark showed modest reordering in the middle and lower tiers, though the methodology underlying these two benchmarks differs enough that direct score comparison between them is unreliable. On SWE-rebench, three models shifted position: Claude Sonnet 4.6 rose from #10 to #10 (no change in rank, though the prior data lists it at 47.2 on Artificial Analysis versus 51.3% here, suggesting score drift or evaluation variance), Gemini 3.1 Pro Preview moved from #9 to #11, and GLM-5.1 jumped from #23 to #12, gaining 10.5 percentage points on Artificial Analysis (from 40.2 to 50.7%). GLM-4.7 similarly advanced 4.4 points on SWE-rebench (33.8 to 38.2) and on Artificial Analysis (33.8 to 38.2), indicating consistent gains. On Artificial Analysis, minor reordering occurred around rank 190 where Magistral Medium 1 and Mistral Medium 3 swapped positions at the 12.5 point level, and at rank 360-362 where three models at 2.7 points reordered. The lack of substantial movement in either benchmark's top ranks suggests stable performance hierarchies, though the gains by GLM models warrant attention to whether they reflect genuine capability improvements or evaluation sensitivity differences between benchmarks.

Cole Brennan