The Inference Report

On the SWE-rebench coding benchmark, the top tier remains stable with OpenAI's gpt-5.5-2026-04-23-xhighModel holding 62.7% (±0.91%), followed by JunieAgent at 61.6% (±0.64%) and OpenAI's CodexAgent at 60.4% (±1.37%), unchanged from the previous round. The Artificial Analysis benchmark, by contrast, shows material reshuffling across its 398-model roster: Claude Fable 5 enters at #1 with 59.9 points, displacing GPT-5.5 to #3, while Claude Sonnet 5 debuts at #5 with 53.4 points, pushing prior entries down. Lower in the Artificial Analysis rankings, DeepSeek V3 climbs from #220 (10.4) to #180 (14.2), a 3.8-point gain that suggests either improved evaluation conditions or a correction in prior assessment. Qwen3.5 9B drops from #101 (25) to #120 (21.4), a 3.6-point decline that warrants scrutiny of methodology consistency. The SWE-rebench benchmark's tight confidence intervals (mostly sub-1.5%) and static ordering suggest a well-controlled experimental setup, whereas Artificial Analysis's broader movement and new entrants indicate either looser evaluation criteria or frequent model updates that shift relative standing. Neither benchmark shows the methodological transparency needed to distinguish between genuine performance improvement and variance in test conditions.

Cole Brennan