The Inference Report

The SWE-rebench rankings show no movement from previous results, with OpenAI gpt-5.5-2026-04-23-xhighModel holding 62.7% at the top, followed by JunieJunieAgent at 61.6% and OpenAI CodexAgent at 60.4%, all within their reported confidence intervals. The Artificial Analysis benchmark, however, reveals substantial reordering across its 408-model field: Claude Fable 5 now leads at 59.9 (previously #1 Claude Fable 5), GPT-5.6 Sol ranks #2 at 58.9 (up from #2 GPT-5.6 Sol), and Kimi K3 holds #3 at 57.1 (unchanged), while deeper in the list models like Mistral Medium 3.1 and Mistral Small 3.1 swapped positions at #181 and #182 with identical 14.7 scores, and QwQ 32B and Qwen3 235B A22B shifted between #198 and #199 at 13.4. These minor reorderings within the Artificial Analysis benchmark reflect typical variance at lower score ranges where models cluster closely, though the methodology for this benchmark remains opaque compared to SWE-rebench's explicit task-based evaluation. The SWE-rebench stability across all 24 models tested suggests either genuine performance plateau or insufficient evaluation frequency to detect meaningful drift in agent-based coding tasks. Without prior Artificial Analysis snapshots provided, the significance of its reordering cannot be assessed against historical patterns.

Cole Brennan