The Inference Report

Claude Opus 4.6 moved to the top of the SWE-rebench rankings at 65.3%, up from fourth place at 53% on Artificial Analysis, while Gemini 3.1 Pro Preview fell from a tie for first at 57.2 to fifth at 62.3% on the coding benchmark, and GLM-5 climbed from seventh at 49.8 to third at 62.8%. The SWE-rebench scores show tighter clustering in the top tier, the gap between first and fifth is only 3 percentage points, compared to the Artificial Analysis benchmark where GPT-5.4 and Gemini 3.1 Pro Preview tied at 57.2, suggesting the coding task may be more discriminative or the models' relative strengths differ meaningfully between general reasoning and software engineering. Kimi K2.5 advanced from sixteenth at 46.8 on Artificial Analysis to thirteenth at 58.5% on SWE-rebench, and Kimi K2 Thinking jumped from thirty-seventh at 40.9 to seventeenth at 57.4%, indicating these models have particular strength in code generation tasks. The SWE-rebench benchmark itself lacks published methodology details in the data provided, no information on test set size, task distribution, evaluation criteria, or whether results are from initial release or continued refinement, making it difficult to assess whether the ranking shifts reflect genuine capability differences or methodological divergence from Artificial Analysis.

Cole Brennan