The Inference Report

The SWE-rebench rankings remain unchanged from the previous snapshot, with OpenAI's gpt-5.5-2026-04-23-xhighModel holding first place at 62.7% ± 0.91%, followed by JunieJunieAgent at 61.6% ± 0.64% and OpenAICodexAgent at 60.4% ± 1.37%. The confidence intervals overlap substantially across the top tier, suggesting the performance gap between positions one through five is within measurement noise. Artificial Analysis data shows broader dispersion across 408 models, with Claude Fable 5 leading at 59.9 and models dropping below 2% in the long tail, but this benchmark uses different evaluation methodology and does not track software engineering task completion in the same way as SWE-rebench. The absence of movement in the SWE-rebench leaderboard across two reporting cycles could indicate either genuine stability in the coding agent landscape or insufficient statistical power to detect meaningful shifts given the modest confidence intervals. Without prior historical snapshots beyond these two identical readings, it remains unclear whether the SWE-rebench results represent a genuine plateau in agent performance or simply reflect the inherent variance of the evaluation itself.

Cole Brennan