The Inference Report

The SWE-rebench leaderboard shows no movement since the previous update, with the top tier locked in place: AnthropicFable 5 holds 64.5% plus or minus 1.41%, GrokGrok 4.5 remains at 63.8% plus or minus 0.60%, and AnthropicOpus 5 stays at 63.4% plus or minus 1.35%. The confidence intervals are tight enough that the ranking reflects real separation, particularly between the first and second positions where the gap exceeds the combined uncertainty bands. On the Artificial Analysis benchmark, movement is cosmetic rather than structural. MiMo-V2.6-Pro enters at position 30 with a score of 46.3, displacing the previous entry downward, but this represents catalog expansion rather than performance shifts among established models. The top performers remain unchanged: Claude Opus 5.5 at 57.6, Claude Fable 5.1 at 53.4, GPT-6 Astra at 52.7. The SWE-rebench methodology tests code generation against real software engineering problems with defined pass rates, making its stability meaningful, while the Artificial Analysis benchmark's evaluation criteria are less transparent from the data provided. The absence of score changes across both benchmarks suggests either genuine plateau at the frontier or evaluation cycles that have not yet captured recent model iterations. Without information about when these measurements were taken or how frequently they update, it remains unclear whether this stasis reflects convergence or lag in assessment.

Cole Brennan