The Inference Report

The SWE-rebench results show no movement since the previous cycle: AnthropicFable 5 holds position one at 64.5% with a confidence interval of 1.41%, followed by GrokGrok 4.5 at 63.8% and AnthropicOpus 5 at 63.4%, identical to prior rankings. The Artificial Analysis leaderboard exhibits modest churn across its 442-entry roster, with two new entrants, Granite 4.2 30B at position 142 (23.7) and Agnes 2.5 Pro Beta at position 29 (49.1), displacing models that previously held those slots. The coding benchmark's stability across top performers suggests either convergence toward a ceiling or insufficient temporal resolution to detect meaningful gains; the tight error bars (0.54 to 1.83 percentage points) indicate the measurements themselves are reliable, but the lack of differentiation between cycles raises a question about whether SWE-rebench remains sensitive to model improvements at this performance tier. Conversely, the Artificial Analysis benchmark's broader movement pattern, particularly the entry of newer variants like Granite 4.2 and Agnes 2.5 Pro Beta, implies that general-capability benchmarks continue to register iterative progress across the field, though the magnitude of individual shifts remains modest and the methodology underlying Artificial Analysis scores is opaque, making it difficult to assess whether the observed reordering reflects genuine capability changes or variance in evaluation conditions.

Cole Brennan