The Inference Report

Claude Opus 4.6 holds the top position on SWE-rebench at 65.3%, unchanged from the previous measurement, while the Artificial Analysis benchmark shows no structural movement in the top tier either, with GPT-5.5 remaining at 60.2 and Claude Opus 4.6 fixed at 53. The most notable shifts occur in the middle tiers of the Artificial Analysis leaderboard, where DeepSeek V4 Pro enters at rank 12 with 51.5, a new entry that suggests incremental capability expansion in the reasoning-focused model space, though the score itself represents performance consistent with existing peers rather than a departure. On SWE-rebench, models like GLM-5 (62.8%), gpt-5.4-2026-03-05-medium (62.8%), and GLM-5.1 (62.7%) cluster tightly in the 62 to 63 percent range, indicating a plateau in absolute gains at the frontier, where further differentiation requires sub-point precision. Gemini 3.1 Pro Preview dropped from rank 3 to rank 6 on SWE-rebench (57.2 to 62.3%), a 4.9-point decline that warrants scrutiny into whether the evaluation conditions or test set composition shifted, as such movements in established models typically signal methodology changes rather than genuine capability loss. The Artificial Analysis benchmark, which tracks a broader set of models, shows no entries with scores above 60.2, creating a visible gap between the two benchmarks' top performers that suggests they may be measuring different problem distributions or difficulty profiles; SWE-rebench appears to emphasize code generation under constraints that newer models handle more effectively, while Artificial Analysis may weight reasoning or multi-step tasks more heavily. Neither benchmark exhibits the velocity that would indicate a meaningful breakthrough, and the lack of new entrants at the very top suggests the field is consolidating rather than expanding capability frontiers.

Cole Brennan