The Inference Report

The SWE-rebench rankings show no movement from the previous snapshot: OpenAI's gpt-5.5-2026-04-23-xhighModel holds first at 62.7% ± 0.91%, followed by JunieAgent at 61.6% ± 0.64% and CodexAgent at 60.4% ± 1.37%, with the remaining top 17 entries unchanged in both position and score. The confidence intervals are narrow enough that these represent genuine separations, not measurement noise, and the xhigh configuration of gpt-5.5 shows a clear advantage over its medium variant at position five (58.9%). Artificial Analysis, by contrast, presents a different ranking entirely, with Claude Fable 5 leading at 59.9, followed by Claude Opus 4.8 at 55.7 and GPT-5.5 at 54.8. The two benchmarks diverge substantially in their top performers: SWE-rebench favors agentic systems and OpenAI variants, while Artificial Analysis ranks Claude models more prominently. This divergence likely reflects methodological differences between the benchmarks. SWE-rebench's focus on software engineering tasks with agentic scaffolding may reward systems optimized for tool use and iterative problem-solving, whereas Artificial Analysis may weight different capabilities or evaluation protocols. The stability of SWE-rebench across this snapshot suggests the frontier has plateaued momentarily, or updates are not yet reflected in this data. Without historical comparison points, it remains unclear whether this stasis is typical or represents genuine convergence in model performance on the task.

Cole Brennan