The Inference Report

Claude Opus 4.6 has moved from fourth to first place on SWE-rebench, reaching 65.3% after climbing 12.3 percentage points from its prior 53% on Artificial Analysis, while Gemini 3.1 Pro Preview has dropped from the top position to sixth at 62.3%, down 4.9 points from its previous 57.2%. The top tier shows compression rather than separation: positions two through five cluster between 62.7% and 64.4%, with gpt-5.2-2025-12-11-medium at 64.4%, GLM-5 and gpt-5.4-2026-03-05-medium both at 62.8%, and GLM-5.1 at 62.7%. Mid-tier volatility appears more pronounced, with Kimi K2.5 jumping from 20th (46.8) to 16th (58.5%), Kimi K2 Thinking climbing from 42nd (40.9) to 21st (57.4%), and GLM-4.7 surging from 34th (42.1) to 14th (58.7%), suggesting these models either benefited from task-specific improvements or the benchmark methodology shifted to reward their particular strengths. The Artificial Analysis rankings show smaller absolute movements across the full list, with most models holding positions within a few slots, which raises a question about whether SWE-rebench and Artificial Analysis are measuring overlapping but distinct problem spaces or whether one benchmark has higher variance than the other. Without clarity on whether SWE-rebench underwent evaluation changes or the models received updates, the magnitude of these shifts makes it difficult to separate genuine capability improvements from benchmark drift.

Cole Brennan