The Inference Report

The SWE-rebench leaderboard shows no movement at the top nine positions, with AnthropicFable 5 maintaining 64.5% and the tier below it unchanged through OpenAICodexAgent at 58.0%. The consistency across these scores, identical to the previous snapshot, reflects either a stabilization in performance or a lack of new evaluations on this benchmark. The confidence intervals remain tight enough that the ranking would require meaningful score shifts to alter, particularly in the 0.5-1.4 percentage point range where several models cluster. On Artificial Analysis, which measures a broader set of coding tasks, the ordering shows minor churn in the 45-67 position band, with Apodex 1.1 dropping from #47 to #66 on a score decline from 30.4 to 26.4, while models like Claude Sonnet 4.6 and Gemini 3.1 Pro Preview each gained one position. These shifts are modest and do not reflect fundamental reordering of capability tiers. The divergence between benchmarks is notable: models ranked highest on SWE-rebench (Fable 5, Grok 4.5, Opus 5) occupy positions 5, 27, and 1 respectively on Artificial Analysis, suggesting the benchmarks measure different aspects of coding performance or that SWE-rebench may emphasize repository-level software engineering tasks where agentic orchestration matters more than raw instruction-following ability. Without methodological transparency on how SWE-rebench constructs its evaluation, particularly whether it tests tool use, multi-step reasoning, or purely code generation, the stability of its rankings is difficult to interpret as either meaningful stasis or measurement artifact.

Cole Brennan