The Inference Report

On SWE-rebench, the coding agent benchmark, the top rankings show no movement: AnthropicFable 5 holds first place at 64.5% ± 1.41%, followed by GrokGrok 4.5 at 63.8% ± 0.60% and AnthropicOpus 5 at 63.4% ± 1.35%, with confidence intervals that do not overlap meaningfully across the top five positions. The Artificial Analysis benchmark tells a different story, where two new entries appear in the top tier: Qwen3.8 2.4T A95B enters at #7 with 57.7, pushing Claude Opus 4.8 down one position, and Gemini 3.7 Flash debuts at #12 with 56.0. The broader rankings remain largely stable across 432 positions, with most models retaining their previous slots or shifting by single positions. The consistency across SWE-rebench contrasts with modest churn in Artificial Analysis, where new models occasionally displace prior entrants but without dramatic reshuffling. Neither benchmark shows the kind of compression at the top that would indicate a breakthrough in reasoning or coding capability; instead, the data reflects incremental refinement and occasional new model releases entering established hierarchies. The stability of the SWE-rebench top ten, despite its larger confidence intervals, suggests the evaluation has reached a plateau where further gains require architectural innovation rather than parameter tuning.

Cole Brennan