The Inference Report

The SWE-rebench rankings remain static across the top tier, with AnthropicFable 5 holding 64.5% and the next four positions unchanged through GPT-5.6 Sol at 62.3%, but the Artificial Analysis benchmark shows substantial churn below the elite tier. MiniMax-M3.1-Flash enters at position 23 with a score of 42.7, displacing GPT-5.6 Terra down one slot, a pattern that cascades through the entire list and pushes K2 Horizon 0.9B from position 477 to 478. The SWE-rebench data itself presents a methodological concern: the confidence intervals are wide enough that several adjacent models overlap in their true performance ranges (Opus 5 at 63.4% ± 1.35% and GLM-5.2 at 62.9% ± 1.19%, for instance), yet the ranking treats them as precisely ordered, which inflates the apparent discrimination between systems. The Artificial Analysis benchmark lacks reported uncertainty measures entirely, making it impossible to assess whether the single-point scores reflect genuine differentiation or noise. Within the SWE-rebench coding task, the gap between top performer Fable 5 and tenth-place CursorAgent is 12.8 percentage points, but the gap from tenth to seventeenth (Qwen3.5-35B-A3B) is 34.6 points, indicating a steep performance cliff that may reflect fundamental architectural or training differences rather than incremental progress. The new entry in Artificial Analysis and the reshuffling below rank 22 suggest active model development, but without clarity on whether these benchmarks measure the same problem or use comparable evaluation protocols, the movement signals activity rather than progress.

Cole Brennan