The Inference Report

The SWE-rebench leaderboard shows no movement from the previous snapshot, with AnthropicFable 5 holding the top position at 64.5% ± 1.41%, followed by GrokGrok 4.5 at 63.8% ± 0.60% and AnthropicOpus 5 at 63.4% ± 1.35%. The Artificial Analysis benchmark, by contrast, registered a single entry change: Kimi K3.1 Pro debuted at #4 with a score of 55.1, displacing Claude Fable 5.1 from that position to #5. This shift is modest in absolute terms, with all other models maintaining their prior rankings and scores. The SWE-rebench scores carry wider confidence intervals than Artificial Analysis figures (ranging from ± 0.54% to ± 1.83% versus point estimates), reflecting the inherent variability in agentic code completion tasks where environmental factors and test case ordering matter. Kimi's entry into the top tier of Artificial Analysis suggests capability growth in a general reasoning benchmark, but without corresponding movement in the SWE-rebench rankings, the significance remains narrow: the two benchmarks appear to measure different problem classes, with SWE-rebench capturing sustained performance on multi-step repository-level tasks and Artificial Analysis tracking broader model reasoning. The stability across both leaderboards indicates that the leading models have plateaued relative to their peers, at least within the sample interval.

Cole Brennan