The Inference Report

Claude Opus 4.6 has moved from fourth place on the SWE-rebench to first with a 65.3% pass rate, a 12.3-point gain from its previous 53% on Artificial Analysis, while Gemini 3.1 Pro Preview dropped from second to fifth despite scoring 62.3% on the coding task, suggesting the two benchmarks measure different problem distributions or evaluation criteria. GLM-5 and Kimi K2.5 both climbed sharply on SWE-rebench, with the latter jumping from 46.8% to 58.5%, and Kimi K2 Thinking advanced from 40.9% to 57.4%, indicating that reasoning-oriented model variants are gaining traction on software engineering tasks. The SWE-rebench top tier shows tighter clustering around 60-65% than the Artificial Analysis rankings, where the gap between first and fifth is only 4.9 points compared to 2.4 points in the older benchmark, suggesting SWE-rebench may be more discriminative among capable models or that recent releases have converged on similar capabilities for this task class. The benchmark methodology appears to emphasize practical code generation and repository-level problem-solving rather than general reasoning or mathematical ability, given that models like gpt-5.4 rank higher on Artificial Analysis (57.2) than on SWE-rebench (62.8 for gpt-5.4-2026-03-05-medium), indicating task-specific tuning or architectural choices matter more than raw parameter count or general reasoning prowess.

Cole Brennan