The Inference Report

The SWE-rebench rankings show no movement from the previous cycle, with OpenAI's gpt-5.5-2026-04-23-xhighModel holding the top position at 62.7% ± 0.91%, followed by JunieJunieAgent at 61.6% ± 0.64% and OpenAICodexAgent at 60.4% ± 1.37%. The confidence intervals across the top performers remain wide enough that several models within the top ten overlap in their true performance ranges, particularly Claude CodeAgent (59.6% ± 1.98%) and the medium variant of gpt-5.5 (58.9% ± 0.78%), suggesting that ranking precision at these levels may exceed what the benchmark's variance actually supports. The Artificial Analysis benchmark, by contrast, shows substantial churn in the lower ranks: Nemotron Cascade 2 30B A3B dropped from position 131 to 164, while models in the 130-165 range shuffled significantly, yet the top tier remains stable with Claude Fable 5 at 59.9 and GPT-5.6 Sol at 58.9. The absence of movement in SWE-rebench's top positions across evaluation cycles, combined with overlapping error bars, suggests either genuine plateau in agent performance on this benchmark or that the test set may be approaching saturation for the leading approaches. The Artificial Analysis churn in lower positions reflects typical ranking volatility among closely-scored models rather than meaningful capability shifts.

Cole Brennan