The Inference Report

On SWE-rebench, the rankings remain stable with AnthropicFable 5 holding first place at 64.5% and confidence intervals narrow enough to distinguish the top tier from the rest, though the spread between positions one and five spans only 2.2 percentage points within overlapping error margins, making the practical gap between the highest performers modest. Artificial Analysis shows more movement: Claude Fable 5.2 enters at the top with 64.6, displacing Gemini 4 Carbon from first to second, and two new entries appear in the lower rankings (Muse Ember at position 17 and gpt-oss-120b at position 62), though these additions do not alter the standing of most existing models. The SWE-rebench benchmark measures code generation on real pull requests with controlled evaluation conditions, making it methodologically stronger than Artificial Analysis for assessing engineering capability, yet the consistency of top performers across both benchmarks suggests the ranking reflects genuine capability differences rather than benchmark-specific artifacts. Neither dataset shows the kind of discontinuous leap that would indicate a methodological shift or a fundamentally new approach to the problem; instead, the movement reflects incremental refinement and the introduction of new model variants within established families.

Cole Brennan