The Inference Report

Claude Opus 4.6 moved from position 7 to 1 on the SWE-rebench with a gain of 12.3 percentage points (53% to 65.3%), while GLM-5 and GLM-5.1 climbed from positions 14 and 11 to 3 and 5 respectively, each gaining over 13 points. On the Artificial Analysis benchmark, GPT-5.5 entered at the top with 60.2 points, displacing Claude Opus 4.7 from first place despite that model holding an identical score of 57.3; Gemini 3.1 Pro Preview remained at 57.2, and the top tier compressed significantly with minimal movement among established leaders. The SWE-rebench shows concentrated gains in the 50-65% range where models demonstrate meaningful progress on real repository tasks, though the methodology does not clearly specify whether these represent improvements in the same test set or refreshed evaluation data. The Artificial Analysis leaderboard exhibits dense clustering and wholesale position shifts despite unchanged scores for many models, suggesting either score rounding or reranking by secondary criteria rather than actual performance changes. The divergence between these two benchmarks is notable: Claude Opus 4.6 dominates SWE-rebench but ranks only 8th on Artificial Analysis at 53, while GPT-5.5 tops Artificial Analysis but does not appear in the SWE-rebench top 34, indicating the benchmarks measure different capabilities or that their evaluation protocols operate on substantially different distributions. Without documentation of evaluation scope or date, it is unclear whether these movements reflect genuine capability gains, model updates, or methodological shifts.

Cole Brennan