The Inference Report

The SWE-rebench rankings remain frozen at their previous positions, with AnthropicFable 5 holding first place at 64.5% ± 1.41%, followed by GrokGrok 4.5 at 63.8% ± 0.60%, and AnthropicOpus 5 at 63.4% ± 1.35%, suggesting either a measurement plateau or a pause in the release cadence for coding agents. On the Artificial Analysis benchmark, the roster has shifted substantially: Claude Opus 5 now leads at 63.1 (up from second), while Claude Fable 5 dropped to 62.1 (down from first), and a new entry, DeepSeek V4 Flash Vision at 51.5, has inserted itself at position 26, pushing all subsequent models down one slot. The divergence between these two benchmarks is notable, SWE-rebench evaluates coding task completion in a controlled sandbox environment with reproducible test suites, while Artificial Analysis appears to measure general capability across a broader evaluation matrix, which explains why the same models rank differently across the two tests. The stability in SWE-rebench contrasts with the churn in Artificial Analysis, where the top tier remains competitive but entries below position 25 experience consistent downward pressure as newer variants appear. Neither benchmark methodology is disclosed in detail here, but the consistency of SWE-rebench's top performers and their tight clustering (all within 8.3 percentage points) suggests the task distribution may be saturating for frontier models, while Artificial Analysis's wider spread indicates greater differentiation across capability profiles.

Cole Brennan