The Inference Report

The SWE-rebench and Artificial Analysis rankings show stability at the top but meaningful movement in the middle tier. On SWE-rebench, the top six positions remain unchanged: gpt-5.5-2026-04-23-xhigh leads at 62.7%, followed by Junie at 61.6%, Codex at 60.4%, Claude Code at 59.6%, gpt-5.5-2026-04-23-medium at 58.9%, and Claude Opus 4.8-xhigh at 56.5%. The notable shifts occur below this ceiling. Claude Sonnet 4.6 climbed from position 10 with 47.2% to position 10 with 51.3%, a 4.1-point gain; Gemini 3.1 Pro Preview rose from position 9 at 46.5% to position 11 at 51.1%, a 4.6-point increase; GLM-5.1 jumped from position 23 at 40.2% to position 12 at 50.7%, an extraordinary 10.5-point improvement; and GLM-4.7 advanced from position 51 at 33.8% to position 17 at 38.2%, a 4.4-point gain. Gemini 3.5 Flash, conversely, declined from position 7 at 50.2% to position 13 at 49.5%. These movements suggest either benchmark variance or genuine performance shifts in the middle tier, though GLM-5.1's dramatic rise warrants scrutiny of whether the test conditions or model capability changed materially. Artificial Analysis rankings remain consistent across the top 100 positions with only minor reordering among tied scores in the 6 to 7-point range, indicating more stable evaluation methodology or less volatility in that benchmark's test set.

Cole Brennan