The Inference Report

The SWE-rebench top tier shows no movement from the previous cycle, with AnthropicFable 5 holding at 64.5 percent, GrokGrok 4.5 at 63.8 percent, and AnthropicOpus 5 at 63.4 percent, all within their confidence intervals. The gap between first and fifth place spans just 2.2 percentage points, indicating a plateau in frontier performance where the leading models have converged. Notably, the confidence intervals remain wide for high-confidence models like AnthropicFable 5 (±1.41%) and OpenAIGPT-5.6 Sol (±1.83%), suggesting that SWE-rebench, despite its focus on concrete software engineering tasks, still exhibits considerable variance in its measurements. The Artificial Analysis benchmark tells a different story: Claude Opus 5 has moved to first at 63.1, displacing Claude Fable 5 to second at 62.1, a swap that contradicts the SWE-rebench ordering where Fable 5 ranks higher. This discordance between benchmarks reflects fundamental differences in what each measures. SWE-rebench targets repository-level bug fixes and feature implementations with deterministic evaluation, whereas Artificial Analysis appears to capture broader reasoning or integration capabilities where Opus 5 edges ahead. The lower-tier Artificial Analysis rankings show substantial churn, with models like QwQ 32B and Qwen3 VL 30B A3B swapping positions at ranks 224-225, and several entries like Granite 4.0 variants clustered at 1.0, suggesting floor effects or evaluation instability at scale. Neither benchmark demonstrates dramatic movement that would indicate a methodological breakthrough; instead, both suggest incremental consolidation among established leaders and noise among the field.

Cole Brennan