The Inference Report

The SWE-rebench leaderboard shows no movement in the top positions; AnthropicFable 5 remains at 64.5% plus or minus 1.41%, with GrokGrok 4.5 and AnthropicOpus 5 holding their second and third positions at 63.8% and 63.4% respectively. The confidence intervals are wide enough that these three models occupy a statistical plateau where ranking shifts would require changes well outside current uncertainty bounds. Below the top tier, the spread widens noticeably: Z.aiGLM-5.2 sits at 62.9% plus or minus 1.19%, OpenAIGPT-5.6 Sol at 62.3% plus or minus 1.83%, and JunieJunieAgent at 61.8% plus or minus 0.54%, indicating that agents with tighter confidence intervals (like Junie) may have undergone more stable evaluation conditions than models with larger error margins. The Artificial Analysis benchmark presents a different picture, with Claude Opus 5 leading at 63.1 rather than Fable 5, suggesting the two benchmarks measure or weight code-solving performance differently or that the models respond differently to their evaluation protocols. The SWE-rebench methodology appears to favor Anthropic's Fable variant and Grok 4.5 specifically, while Artificial Analysis ranks Opus higher, a divergence worth noting when interpreting which benchmark better predicts real coding task performance. Neither benchmark shows dramatic velocity in the upper ranks, which either reflects genuine saturation on the problem set or indicates that incremental improvements at this performance ceiling require substantially larger modeling or data investments than the recent release cycle has provided.

Cole Brennan