The Inference Report

The SWE-rebench leaderboard shows no movement since the previous update, with the same twenty-four models holding identical positions and scores, while the Artificial Analysis benchmark exhibits extensive shuffling across its four-hundred-plus entries. On the coding task benchmark, OpenAI's gpt-5.5-2026-04-23-xhighModel maintains 62.7% (±0.91%), Junie's agent holds 61.6% (±0.64%), and the top six models cluster between 56.5% and 62.7% with confidence intervals typically under two percentage points, suggesting stable measurement but limited differentiation in the high-performing tier. The Artificial Analysis data presents a different picture: Hy3 enters at position 26 with 41.2, KAT Coder Pro V2 drops from 35.4 to 33.7 and falls from rank 55 to 61, and models throughout the middle ranks shift by several positions without corresponding score changes, indicating the movements reflect reordering rather than performance updates. This pattern of rank reshuffling without score modification in Artificial Analysis suggests either a change in how ties are broken or a recomputation of model ordering logic, whereas the frozen SWE-rebench results offer no evidence of methodological change or new model evaluation since the previous brief. The confidence intervals on SWE-rebench remain tight enough to rule out noise, but the complete stasis across the leaderboard warrants clarification about whether these represent the same evaluation run or whether the benchmark itself has paused.

Cole Brennan