The Inference Report

On the SWE-rebench coding benchmark, the top tier remains stable with OpenAI's gpt-5.5-2026-04-23-xhighModel holding first at 62.7% ± 0.91%, followed by JunieAgent at 61.6% ± 0.64% and OpenAI CodexAgent at 60.4% ± 1.37%, a ranking that shows no movement from the previous cycle. The confidence intervals are reasonably tight for the leading models, with the xhigh variant of gpt-5.5 maintaining a 1.1-point gap over Junie, though the uncertainty bands overlap slightly. Below the top three, Claude CodeAgent (59.6% ± 1.98%) and the medium variant of gpt-5.5 (58.9% ± 0.78%) round out the top five with no repositioning. The methodology here appears sound for the leading entries, as these are agent-based systems evaluated on real software engineering tasks where the error bars reflect genuine variance in performance across different problem types. Further down the ranking, models like Claude Opus 4.8-xhigh (56.5% ± 1.20%) and Claude Opus 4.7-high (53.1% ± 1.45%) show larger confidence intervals, suggesting less consistent behavior across the test suite. The Artificial Analysis benchmark tells a different story, with Claude Fable 5 now ranking first at 59.9, displacing GPT-5.6 Sol to second at 58.9, and Kimi K3 entering at third with 57.1, pushing Claude Opus 4.8 down to fourth at 55.7. The entry of Kimi K3 and the reordering of top positions suggests genuine capability shifts rather than noise, though the Artificial Analysis scores lack confidence intervals entirely, making it difficult to assess whether these movements reflect real performance differences or measurement variability. The absence of error bars on the Artificial Analysis data is a methodological weakness that limits confidence in the ranking movements, particularly when comparing models separated by less than a point.

Cole Brennan