The Inference Report

The SWE-rebench rankings show no movement at all: AnthropicFable 5 remains at 64.5 percent, GrokGrok 4.5 at 63.8 percent, and the top ten hold their positions with identical scores and confidence intervals to the previous cycle. This stability across all seventeen entries on the coding benchmark suggests either that the evaluation window was too brief to register meaningful change or that the models tested have reached a plateau on this particular task distribution. The Artificial Analysis benchmark tells a different story, with three new entries (Claude Opus 5 at position 11, Tencent Hy4 at 17, and Grok 4.6 at 22) displacing prior models and creating cascading rank shifts through the lower positions, though the top ten remain largely intact with Gemini 4 Carbon holding the lead at 60 points. The divergence between benchmarks is notable: SWE-rebench, which evaluates agentic code-solving on real repositories, shows zero churn in its top tier, while Artificial Analysis, a broader evaluation framework, registers turnover in the mid-range. Without details on the evaluation methodology changes between cycles, it is unclear whether SWE-rebench's stability reflects genuine performance parity among top models or whether its test set has become less discriminative at the frontier. The confidence intervals on SWE-rebench (ranging from 0.54 to 1.83 percentage points) are wide enough that several apparent ranking differences fall within noise, which may explain why no reordering occurred despite ongoing model development.

Cole Brennan