The Inference Report

The SWE-rebench rankings remain frozen at their previous positions, with no movement across the top 17 coding agents. OpenAI's gpt-5.5-2026-04-23-xhighModel holds first at 62.7% (±0.91%), followed by JunieAgent at 61.6% (±0.64%), and the spread narrows predictably through the field. The confidence intervals are tight enough to distinguish most placements, though Claude Opus 4.6-high (47.8% ±1.37%) and Claude Sonnet 4.6 (51.3% ±0.55%) overlap slightly at the boundary of statistical noise. Across the Artificial Analysis benchmark, the data shows substantial churn in the middle and lower tiers, Magistral Medium 1.2 dropped from position 130 to 148, while Apriel-v1.6-15B-Thinker moved from 129 to 128, but the top performers remain locked in place: Claude Fable 5 leads at 59.9, with GPT-5.5 and Claude Opus 4.8 holding their second-tier positions at 54.8 and 55.7 respectively. The two benchmarks measure different problem spaces (SWE-rebench targets repository-level software engineering tasks while Artificial Analysis covers broader reasoning), which explains why their orderings diverge: coding-specific systems like JunieAgent rank higher on SWE-rebench but Claude Fable 5 tops the general benchmark. Without prior Artificial Analysis scores, it is unclear whether the observed shuffling in positions 128 to 148 reflects genuine performance changes or measurement variance. The stability in SWE-rebench suggests the top agents have reached a plateau, or that the evaluation's resolution cannot detect sub-point improvements.

Cole Brennan