The Inference Report

Claude Opus 4.6 maintains its lead on SWE-rebench at 65.3 percent, a 12.3-point gain from its prior position at 53 percent on the Artificial Analysis benchmark, though the two evaluations measure different problem spaces and should not be treated as directly comparable. The top tier has consolidated around 62 to 65 percent on SWE-rebench, with gpt-5.2-2025-12-11-medium at 64.4 percent and GLM-5 and gpt-5.4-2026-03-05-medium both at 62.8 percent; these four models now occupy the summit, separated by narrow margins. Movement in the broader field reveals uneven progress: Kimi K2 Thinking jumped from position 37 at 40.9 percent to position 17 at 57.4 percent, a 16.5-point increase suggesting either a model update or a shift in how the benchmark evaluates reasoning-focused architectures, while Kimi K2.5 advanced from position 16 at 46.8 percent to position 13 at 58.5 percent. Gemini 3 Flash Preview climbed from position 18 at 46.4 percent to position 22 at 52.5 percent, and Nova 2.0 Lite moved from position 76 at 29.7 percent to position 58 at 34.5 percent on the Artificial Analysis side, though this latter shift reflects reranking rather than SWE-rebench performance. The SWE-rebench scores themselves show no obvious saturation at the top: the gap between first and fifth place is 2.8 percentage points, and the distribution below rank 10 remains steep, suggesting that either the benchmark has sufficient discriminative power or that model capabilities on software engineering tasks continue to stratify sharply by architecture and training approach. What remains unclear from the data alone is whether these gains reflect genuine improvements in code generation and repository-level reasoning, or whether they reflect changes in evaluation methodology, task distribution, or model selection within the rebench suite.

Cole Brennan