The Inference Report

Claude Opus 4.6 holds the top position on SWE-rebench at 65.3%, unchanged from the previous ranking, while the field beneath it shows modest consolidation rather than dramatic reshuffling. The most notable movement comes from Chinese models: GLM-5 jumped from rank 17 to rank 3 (49.8 to 62.8 percent), GLM-5.1 climbed from rank 14 to rank 6 (51.4 to 62.7 percent), and GLM-4.7 advanced from rank 44 to rank 14 (42.1 to 58.7 percent), suggesting systematic improvements in that family's code-solving capability. Kimi K2.5 rose from rank 29 to rank 16 (46.8 to 58.5 percent), and Kimi K2 Thinking moved from rank 54 to rank 21 (40.9 to 57.4 percent), indicating progress across Kimi's lineup as well. Claude Sonnet 4.6 improved from rank 12 to rank 9 (51.7 to 60.7 percent), while Gemini 3.1 Pro Preview declined from rank 3 to rank 7 (57.2 to 62.3 percent), suggesting variable iteration quality. The Artificial Analysis benchmark tells a different story: GPT-5.5 leads at 60.2, with Claude Opus 4.6 at rank 9 (53 points) and Claude Opus 4.7 at rank 2 (57.3 points), revealing substantial disagreement between the two evaluations on which models excel at real-world software engineering tasks. The divergence between these benchmarks, SWE-rebench emphasizing repository-level problem solving and Artificial Analysis potentially capturing different task distributions or evaluation criteria, warrants scrutiny of their respective methodologies before treating either ranking as definitive for model capability assessment.

Cole Brennan