The Inference Report

The SWE-rebench leaderboard shows no movement since the previous update, the top 24 entries remain identical in rank and score, with OpenAI's gpt-5.5-2026-04-23-xhigh still at 62.7% (±0.91%), Junie's agent at 61.6% (±0.64%), and the rest holding steady through the Claude Sonnet 4.6 at 51.3% (±0.55%) and down to Gemma 4 31B at 16.5% (±1.13%). The confidence intervals are tight enough that no model has shifted position within measurement error. By contrast, the Artificial Analysis benchmark shows substantial reshuffling in the lower ranks, models from position 304 onward have reordered themselves, with Gemma 3 12B Instruct climbing from #345 to #304 (3.4 to 5.5), and LFM2.5-1.2B-Thinking and LFM2.5-1.2B-Instruct swapping positions 364 and 366, but the top 50 models remain in nearly identical order, suggesting the two benchmarks measure overlapping but not identical capabilities. The SWE-rebench methodology, which evaluates agent-based code completion on real repository issues, appears more stable than Artificial Analysis across time, though without historical context it is unclear whether this stability reflects genuine plateau or simply infrequent test runs. The lack of motion in the coding benchmark despite ongoing model releases is noteworthy: neither the latest Claude variants nor new reasoning models have displaced the April OpenAI and Junie entries that occupy the summit.

Cole Brennan