The Inference Report

The SWE-rebench rankings remain unchanged from the previous cycle, with Anthropic's Fable 5 holding the top position at 64.5 percent, followed by Grok 4.5 at 63.8 percent and Opus 5 at 63.4 percent; the stability across the top tier reflects the difficulty of moving beyond these performance plateaus, though the confidence intervals suggest room for meaningful separation if the variance decreases. The Artificial Analysis leaderboard, by contrast, shows considerable churn in the middle ranks, with DeepSeek V4 Flash 0731 entering at position 17 with a score of 49.9, displacing the previous entry and pushing Claude Sonnet 4.6 from 17th to 18th, yet this movement involves models scoring within a compressed band between 47 and 50 where differentiation becomes fragile. At the top of Artificial Analysis, Claude Opus 5 leads at 60.7, a position it does not hold on SWE-rebench where it ranks third; this discrepancy points to benchmark sensitivity in methodology or task composition, suggesting that neither leaderboard alone captures the full picture of coding capability. The SWE-rebench data carries reported confidence intervals, which provide some guard against false precision, whereas Artificial Analysis reports point estimates without uncertainty quantification, making it harder to assess whether observed shifts represent genuine performance changes or measurement noise. What stands out is not volatility at the extremes but rather the emergence of a dense cluster of models in the 40 to 50 range on both benchmarks, indicating that the frontier is broadening even as the absolute ceiling remains contested.

Cole Brennan