The Inference Report

Claude Opus 4.6 holds the top position on SWE-rebench at 65.3%, unchanged from the previous ranking cycle, while the middle tier has consolidated with modest gains across multiple models. Comparing the two benchmarks reveals a persistent divergence: SWE-rebench places Claude Opus 4.6 first by 12.3 points, but Artificial Analysis ranks it fourth at 53.0, with GPT-5.4 leading at 57.2. This gap reflects different evaluation methodologies or problem distributions. On SWE-rebench, Kimi K2.5 advanced from position 13 (58.5%) to position 13 (58.5%) with no reported change, though Kimi K2 Thinking climbed from rank 36 at 40.9 on Artificial Analysis to rank 17 at 57.4% on SWE-rebench, a 16.5-point jump suggesting the thinking variant's code-solving capability improved substantially between assessment cycles. GLM-5 similarly rose from rank 7 (49.8 on Artificial Analysis) to rank 3 (62.8% on SWE-rebench), a 13-point absolute gain. Gemini 3.1 Pro Preview dropped slightly from rank 2 to rank 5 on SWE-rebench (62.3%), though it remained competitive on Artificial Analysis at 57.2. The SWE-rebench benchmark itself appears to reward reasoning-capable models more heavily than Artificial Analysis does, since thinking-oriented variants like Kimi K2 Thinking show larger relative gains in the SWE-rebench ordering. Without access to the specific test cases or evaluation protocol differences between benchmarks, the directional pattern is clear: models with explicit reasoning steps perform better on SWE-rebench, while Artificial Analysis may weight inference speed or other factors more equally.

Cole Brennan