The Inference Report

Claude Opus 4.6 now leads the SWE-rebench rankings at 65.3%, a 12.3 percentage point jump from its prior position at 53%, while the broader top tier shows consolidation rather than dramatic reshuffling: gpt-5.2-2025-12-11-medium (64.4%), GLM-5 (62.8%), and Junie (62.8%) occupy positions 2 through 4 with scores that cluster tightly within a 2.5-point band, suggesting the frontier of coding performance has compressed into a narrow range. The movement is meaningful in specific quarters. GLM-5 rose from rank 17 to rank 3, GLM-5.1 climbed from 14 to 6, and Kimi K2.5 advanced from 29 to 16, indicating that Chinese model families are closing gaps on the leaders, while Gemini 3.1 Pro Preview dropped from rank 3 to rank 7 despite holding a respectable 62.3% score. The Artificial Analysis benchmark, however, tells a different story: it shows far less movement at the top, with GPT-5.5 still leading at 60.2 and Claude Opus 4.6 at rank 9 with 53 points, revealing a significant divergence between the two evaluation frameworks. The SWE-rebench results reflect a methodology focused on software engineering tasks with specific, measurable outcomes, whereas the Artificial Analysis scores may weight different problem classes or evaluation criteria. This divergence matters: a model can rank first on one benchmark while placing ninth on another, which suggests that neither benchmark alone captures complete coding capability. The volume of drops from Artificial Analysis rankings (over 100 models removed) without corresponding SWE-rebench entries makes it impossible to determine whether those models genuinely degraded or were simply deprioritized in evaluation cycles, a methodological gap worth noting when interpreting movement as progress.

Cole Brennan