The Inference Report

Claude Opus 4.6 holds the top position on SWE-rebench at 65.3%, a gain of 12.3 points from its prior Artificial Analysis score of 53, while gpt-5.2-2025-12-11-medium sits second at 64.4%, and GLM-5 and gpt-5.4-2026-03-05-medium tie at 62.8%. The SWE-rebench leaderboard shows material reshuffling in the upper tier: Kimi K2.5 climbed from rank 16 (46.8) to rank 13 (58.5), Kimi K2 Thinking jumped from rank 35 (40.9) to rank 17 (57.4), and Gemini 3 Flash Preview moved from rank 18 (46.4) to rank 22 (52.5), all indicating that these models improved substantially on this benchmark. The Artificial Analysis leaderboard, which tracks a different evaluation methodology, remains largely stable in its upper rankings with GPT-5.4 and Gemini 3.1 Pro Preview tied at 57.2, though KAT Coder Pro V2 entered at rank 23 with 43.8 and Nemotron Cascade 2 30B appeared at rank 81 with 27.7. The gap between the two benchmarks' top scores (SWE-rebench's 65.3 versus Artificial Analysis's 57.2) suggests they measure different aspects of model capability or use distinct evaluation criteria; without details on SWE-rebench's methodology relative to Artificial Analysis, it remains unclear whether the higher scores reflect easier test cases, different task distributions, or genuine performance differences on the same underlying problems. The consistency of model ordering within each benchmark indicates both are internally coherent, but the divergence between them argues for caution in treating either as a complete picture of coding ability.

Cole Brennan