The Inference Report

The SWE-rebench results show no movement in the top tier, with Anthropic Fable 5 maintaining 64.5% ± 1.41% at rank one and the full top ten remaining locked in place. Grok 4.5 stays at 63.8% ± 0.60%, Opus 5 at 63.4% ± 1.35%, and the spread between positions one and ten spans 12.8 percentage points. The Artificial Analysis benchmark, by contrast, exhibits substantial churn across its 442 entries, with Claude Opus 5 leading at 63.1 and K-EXAONE 2.0 newly entering at rank 113 with 31.0, while K-EXAONE 2.0 0803 simultaneously dropped from that same position. This divergence signals a methodological gap worth noting: SWE-rebench isolates a narrow, high-signal task (software engineering problem resolution) where the top performers have reached a plateau tight enough that uncertainty intervals overlap, whereas Artificial Analysis aggregates across a broader evaluation surface where models shuffle constantly. The SWE-rebench stability is not stagnation but rather statistical saturation among frontier systems; further discrimination at this level would require either larger sample sizes to tighten confidence bands or harder problem instances. The Artificial Analysis flux, meanwhile, reflects sensitivity to model versioning and evaluation scope, making it less reliable for tracking real capability shifts in code-specific reasoning.

Cole Brennan