The Inference Report

SWE-rebench shows no movement in the top tier, with AnthropicFable 5 holding 64.5% and the next four models within a 2.2 percentage point band, all within their stated confidence intervals. Artificial Analysis reveals a different story: GLM-5.3 enters at rank 6 with 59.5, displacing Qwen3.8 Max to rank 7, a reordering that reflects genuine competition in the 59 to 60 range where four models now cluster. DeepSeek V3.1 Terminus climbs three positions to 104 from 107 with a 0.3-point gain (31.4 vs. 31.1), a marginal improvement that may signal refinement rather than breakthrough. The broader pattern across both benchmarks is consolidation rather than disruption: the top five on SWE-rebench remain unchanged, and Artificial Analysis shows mostly positional shuffles among models scoring below 35, where entry and exit are frequent. Neither benchmark exhibits the kind of score inflation or collapse that would suggest methodological drift, though the divergence between SWE-rebench (which measures code completion on real GitHub issues with stricter evaluation) and Artificial Analysis (which appears to use a different test set or scoring regime) deserves scrutiny. The confidence intervals on SWE-rebench, particularly the ±1.41% for Fable 5, are large enough that small reorderings in the 56 to 64 range would not be statistically meaningful; Artificial Analysis provides no error bounds, making it impossible to assess whether its movements reflect real capability changes or sampling variance.

Cole Brennan