The Inference Report

On the SWE-rebench, the top tier has crystallized around 60-65 percent resolve rates, with Claude Opus 4.6 holding first place at 65.3 percent, followed by gpt-5.2-2025-12-11-medium at 64.4 percent and a cluster of GLM and GPT variants in the 62-63 percent band. The meaningful movement comes from models that have climbed substantially from prior positions: GLM-5 jumped from rank 11 to rank 3 by gaining 13 percentage points (49.8 to 62.8), GLM-4.7 surged from rank 36 to rank 14 with a 16.6-point gain (42.1 to 58.7), and Kimi K2.5 moved from rank 21 to rank 16 by adding 11.7 points (46.8 to 58.5). Gemini 3.1 Pro Preview, however, dropped from rank 2 to rank 6 despite maintaining a competitive 62.3 percent score, suggesting the benchmark has become more discriminating at the high end. The SWE-rebench scores show larger absolute gains across the board compared to the Artificial Analysis benchmark, which could indicate either improved model capabilities in coding tasks or a shift in evaluation methodology, though the data does not specify whether the benchmark itself was recalibrated. The clustering of models between 58 and 62 percent suggests diminishing returns in further optimization, with the gap between first and tenth place now only 5.4 percentage points compared to what would be expected if progress were linear.

Cole Brennan