The Inference Report

The SWE-rebench rankings have been entirely replaced with a fresh cohort of agent-based systems, with OpenAI's gpt-5.5-2026-04-23-xhigh model leading at 62.7% plus or minus 0.91%, followed by JunieAgent at 61.6% and OpenAI CodexAgent at 60.4%, while the Artificial Analysis benchmark shows Claude Fable 5 in the top position at 59.9, a configuration that differs markedly from the previous leadership structure where Claude Fable 5 held first place. The SWE-rebench methodology appears to favor agentic systems with explicit configuration parameters like "xhigh" and "medium," producing notably higher absolute scores than Artificial Analysis reports for comparable models, suggesting the two benchmarks measure different aspects of coding capability or employ divergent evaluation protocols. All seventeen prior SWE-rebench entries have been dropped, indicating either a benchmark refresh, a shift in how systems are tested, or a change in what qualifies for inclusion, though the new cohort maintains consistency in confidence intervals ranging from 0.53% to 1.98%, which is tighter than one might expect given the diversity of approaches. The Artificial Analysis ranking remains largely stable in its upper tiers, with minor reordering and the addition of Devstral 2 moving from position 165 to 140, but without the wholesale replacement seen in SWE-rebench, suggesting the two evaluation frameworks operate on different cadences or criteria. Without prior SWE-rebench data for the new agent entries, it is not possible to determine whether these scores represent genuine capability gains or simply reflect how agent-based systems perform on that particular benchmark relative to the model-only systems previously ranked.

Cole Brennan