The Inference Report

The SWE-rebench leaderboard shows complete stability across the top tier, with AnthropicFable 5 holding 64.5% ± 1.41%, GrokGrok 4.5 at 63.8% ± 0.60%, and AnthropicOpus 5 at 63.4% ± 1.35%, identical to the previous cycle. The confidence intervals are tight enough that these positions reflect genuine performance differences rather than noise, yet the lack of movement suggests either that the test set has reached saturation among frontier models or that incremental improvements are genuinely stalled. The gap between rank 5 (OpenAIGPT-5.6 Sol at 62.3% ± 1.83%) and rank 14 (DeepSeekDeepSeek-V4 Pro at 40.2% ± 1.29%) is substantial and persistent, indicating a clear separation between high-capability and mid-tier systems. On the Artificial Analysis benchmark, the data shows comprehensive churn throughout the ranking, with Ling 3.1 Flash entering at rank 22 as a new entry, pushing all subsequent models down by one position. This suggests Artificial Analysis tracks a broader ecosystem with more frequent model releases, whereas SWE-rebench's frozen leaderboard may reflect either fewer new submissions or the difficulty of breaking into a plateau dominated by three Anthropic variants. Neither benchmark shows evidence of methodological weakness in the raw numbers, though SWE-rebench's lack of movement warrants scrutiny into whether the test remains sensitive to real capability differences or has become a ceiling effect masking progress in specialized domains.

Cole Brennan