The Inference Report

The SWE-rebench rankings show minimal movement at the top tier, with Claude Code holding 52.9% and the next three positions separated by less than 1.2 percentage points, suggesting performance has plateaued in a narrow band where incremental gains demand substantial effort. Below the top tier, volatility emerges: Claude Opus 4.5 dropped from 49.7% on Artificial Analysis to 43.8% on SWE-rebench (a 5.9-point gap that flags a possible methodology divergence between the two benchmarks), while Gemini 3 Pro Preview fell from 48.4% to 46.7%, and Kimi K2.5 contracted from 46.8% to 37.9%, suggesting these models may perform differently on SWE-rebench's specific test distribution or problem types. Conversely, Kimi K2 Thinking climbed 2.9 points from 40.9% to 43.8%, and GLM-4.6 gained 4.6 points from 32.5% to 37.1%, indicating selective improvements in certain architectures. The Artificial Analysis leaderboard itself remained largely stable in its top 20, with new entry mimo-v2-omni appearing at rank 22 and Nanbeige4.1-3B at rank 169, but these additions occupy middle and lower positions where churn is expected. The key signal is not absolute ranking changes but the widening discrepancy between the two benchmarks: models ranking identically on both (like the top five) inspire confidence, while models showing 5+ point spreads warrant scrutiny into whether SWE-rebench is measuring a materially different capability or whether evaluation methodology accounts for the delta.

Cole Brennan