The Inference Report

On SWE-rebench, the top tier remains static: gpt-5.5-2026-04-23-xhigh holds 62.7%, Junie stays at 61.6%, and the Claude and GPT variants occupy positions three through seven without movement. The meaningful shifts occur in the mid-tier, where GLM-5.1 climbed from position 23 at 40.2% to position 12 at 50.7%, a 10.5-point gain that represents the largest jump in the dataset, while GLM-4.7 rose from position 52 at 33.8% to position 17 at 38.2%. Kimi K2.6 advanced from position 16 to position 15, and Claude Sonnet 4.6 moved from position 8 to position 10 despite scoring identically at 51.3%, suggesting ranking adjustments independent of score changes. Across the Artificial Analysis benchmark, the distribution shows far less volatility: Claude Fable 5 leads at 59.9, the top 20 models cluster between 42.8 and 59.9 with mostly preserved rankings, and a new entry (Nex-N2-Pro at 41.0) appears at position 20 while KAT-Coder-Pro V1 jumped 31 positions from 83 to 52 with a 6.3-point improvement from 28.3 to 34.6. The discrepancy between benchmarks is notable: models ranking high on SWE-rebench (gpt-5.5-xhigh, Junie) do not dominate Artificial Analysis, where Claude Fable 5 leads despite placing second on the coding benchmark, suggesting these metrics capture different problem-solving dimensions or that the evaluation methodologies diverge in what they reward. Neither benchmark shows the compression or volatility typical of immature measurement systems, indicating both have stabilized around consistent model orderings, though the absence of methodological detail prevents assessment of whether either captures real capability differences or primarily reflects training data overlap.

Cole Brennan