The Inference Report

Claude Opus 4.6 holds the top position on SWE-rebench with 65.3%, a jump of 12.3 percentage points from its previous ranking of ninth place at 53%, while the rest of the field has compressed considerably at the high end, with positions two through five separated by only 1.6 points. The movement reflects real gains in code generation capability, though the methodology warrants scrutiny: SWE-rebench measures the ability to resolve GitHub issues end-to-end, a task that benefits from reasoning depth and context management rather than pure instruction-following, and the compression at the top suggests these models are approaching saturation on the benchmark's current problem distribution. On Artificial Analysis, the landscape differs markedly, with GPT-5.5 leading at 60.2 but Claude Opus 4.6 placed ninth at 53, indicating that the two benchmarks reward different capabilities or that Artificial Analysis weights broader performance categories beyond code completion. The divergence between benchmarks matters: Claude Sonnet 4.6 ranks ninth on SWE-rebench at 60.7% but only twelfth on Artificial Analysis at 51.7%, suggesting it excels at the specific demands of issue resolution but underperforms on Artificial Analysis's mixed evaluation. GLM-5 climbed from seventeenth to third on SWE-rebench, gaining 13 points, while Kimi K2 Thinking rose from fifty-fourth to twenty-first on the same benchmark with an 16.5-point gain, patterns that suggest targeted improvements in code reasoning. The Artificial Analysis list saw minimal reordering beyond the top tier, with most models holding their positions, indicating either stable model performance or less frequent evaluation updates on that benchmark. What distinguishes this cycle is not a breakthrough in methodology but consolidation: the top performers are now substantially ahead of the middle tier, and the gap between first and tenth place on SWE-rebench spans 4.6 points, a meaningful spread that reflects real differences in how models handle multi-step code tasks.

Cole Brennan