The Inference Report

The SWE-rebench leaderboard shows no movement at the top tier: Claude Code, Junie, and the gpt-5.2 variants hold positions 1 through 5 with scores between 52.9% and 51.0%, unchanged from the prior cycle. Below that band, however, several models experienced substantial rank shifts that expose inconsistencies between the two evaluation sources. Claude Opus 4.5 dropped from position 8 (49.7 on Artificial Analysis) to position 12 (43.8% on SWE-rebench), a gap of 6 percentage points that suggests either a significant performance regression or a methodological divergence between benchmarks. Kimi K2 Thinking climbed 14 positions on SWE-rebench (from 27 to 13) despite logging only a 2.9-point gain (40.9 to 43.8%), while GLM-5 fell from position 7 (49.8 on Artificial Analysis) to position 15 (42.1% on SWE-rebench), a 7.7-point drop. Kimi K2.5 reversed course entirely, sinking from position 12 (46.8 on Artificial Analysis) to position 19 (37.9% on SWE-rebench). These divergences raise questions about benchmark stability: SWE-rebench appears to reward certain architectural choices or fine-tuning strategies that Artificial Analysis does not, yet neither source clarifies whether the gap reflects genuine capability differences or evaluation artifacts. The lack of detailed methodology documentation for either benchmark makes it difficult to assess whether these swings represent real performance variation or measurement drift. At the frontier, the stability of the top five models suggests that the highest-capability systems may be approaching a plateau on this task distribution, while the volatility in the 7 to 20 ranking band indicates that mid-tier models remain sensitive to benchmark design choices.

Cole Brennan