The Inference Report

The SWE-rebench rankings hold steady at the top, with AnthropicFable 5 retaining first place at 64.5% and the next five positions unchanged through GPT-5.6 Sol. The confidence intervals remain wide enough that the top tier shows genuine separation: Fable 5's 1.41% margin and Grok 4.5's 0.60% margin indicate different levels of measurement precision, yet both models occupy their positions with confidence bounds that don't overlap meaningfully with adjacent ranks. Below the top six, however, the data becomes noisier. JunieAgent holds sixth at 61.8% with a tight 0.54% margin, but Claude CodeAgent and OpenAICodexAgent follow at 60.4% and 58.0%, suggesting a drop-off in performance that the error bars don't fully resolve. The Artificial Analysis rankings tell a different story. Claude Opus 5 leads there at 60.7, while AnthropicFable 5 ranks second at 59.9, a reversal of the SWE-rebench hierarchy that points to benchmark-specific strengths rather than universal capability differences. GPT-5.6 Sol appears at position three in Artificial Analysis (58.9) but holds fifth on SWE-rebench (62.3%), indicating the two evaluation frameworks weight code-generation and general reasoning tasks differently. The divergence widens further down: DeepSeek-V4 Pro scores 40.2% on SWE-rebench but only 44.3 on Artificial Analysis, placing it 23 positions lower in the latter ranking. These inversions suggest neither benchmark captures the full picture of model capability, and that agentic code performance (measured by SWE-rebench) does not correlate linearly with broader reasoning benchmarks.

Cole Brennan