The Inference Report

The SWE-rebench leaderboard holds steady at the top with Anthropic Fable 5 maintaining 64.5% ± 1.41%, followed by Grok 4.5 at 63.8% ± 0.60% and Anthropic Opus 5 at 63.4% ± 1.35%, showing no movement in the top tier. Notably, the confidence intervals on SWE-rebench remain tight across the tested models, suggesting controlled evaluation conditions, though the gap between the leader and tenth-place Cursor Agent (51.7% ± 0.84%) spans 12.8 percentage points, indicating clear stratification in code-solving capability. On the Artificial Analysis benchmark, Gemini 4 Argon enters at number five with 52.6 points as a new entrant, displacing prior entries by one position, while GPT-6 Luna climbed from #34 to #33 with a score increase from 37.3 to 38.1. Solar Mini 4 also appears as a new entry at #91 with 24.1 points. The broader Artificial Analysis leaderboard shows tight clustering in the 5 to 10 point range across positions 380 to 450, where parameter count and model size variations produce minimal score differentiation, raising questions about whether that benchmark's resolution distinguishes meaningfully between smaller models. The SWE-rebench results align with the intuition that coding tasks reward architectural sophistication and training data quality more than raw parameters, whereas Artificial Analysis's crowded lower tier suggests saturation or floor effects in its evaluation methodology.

Cole Brennan