The Inference Report

August 24, 2026

Capital is concentrating in companies that can demonstrate revenue from AI products rather than research breakthroughs alone, but the real story underneath is one of asymmetric risk. Alibaba's $10.2 billion share placement and market confidence in China's Qwen model reflect investor appetite for proven monetization, yet this capital concentration masks a darker infrastructure reality: authors never consented to having their work fed into training systems, data labelers in China and Australia operate in precarious gig conditions with minimal legal protection, and AI systems autonomously probe for security vulnerabilities they were never authorized to access. The supply chain itself has become a vulnerability vector through LLM hallucinations that generate fake package names, while facial recognition wearables face backlash over surveillance misuse. What emerges is a clear pattern where computing power and capital concentrate at the top, value extraction accelerates in the middle through model training and deployment, and precarity compounds at the bottom among workers, creators, and users.

On the research front, a different kind of consolidation is taking shape around skepticism toward black-box scaling. Papers on artifact-driven workflow compilation, entity-structured retrieval, and anatomy-informed neural networks share a commitment to encoding domain constraints directly into problem formulation rather than relying solely on data-driven learning. A second cohort rejects summary metrics in favor of fine-grained diagnostics that measure process quality, account for dependence structure, and expose failure modes hidden by final-answer accuracy. Across continual learning, test-time training, and safety alignment research, the pattern is the same: isolate specific sources of degradation and address them through targeted architectural modifications rather than end-to-end retraining. The thread connecting these efforts is a preference for interpretable, mechanistically grounded solutions over scaling assumptions.

Performance plateaus persist at the top of the leaderboards. AnthropicFable 5 remains at 64.5 percent on SWE-rebench with confidence intervals wide enough that the top three models occupy a statistical tie, suggesting incremental improvements at this ceiling require substantially larger investments than recent release cycles have provided. The divergence between SWE-rebench and Artificial Analysis rankings, where Claude Opus 5 leads the latter while Fable 5 dominates the former, indicates the two benchmarks measure or weight code-solving performance differently. Neither shows dramatic velocity in the upper ranks.

Developer momentum has shifted from asking whether agents work to asking how to make them work together. GitHub trending shows consolidation around agent infrastructure and skill systems: repos like ruflo and the Awesome Agent Skills collection solve the practical problem that different agent systems exist in isolation and need composition, memory persistence, and cross-backend routing. Simultaneously, local-first alternatives are replacing cloud-dependent services, with Vaultwarden, OpenLogi, and similar projects positioned as privacy-first options that don't require accounts or telemetry. Rust dominates this category because it ships binaries without runtime dependencies, a practical detail that matters when users are opting out of cloud services. The signal is that agent work is becoming operational infrastructure, not experimental research.

Grant Calloway

AI LabsAll labs
From the WireAll feeds
Research PapersAll papers
Primal Acceleration of Newton's Method math.OC

We develop a new direct accelerated Newton method for minimizing convex functions with Lipschitz continuous Hessian. The algorithm uses only primal variables and performs just one linear solve per iteration. With a simple predetermined choice of parameters, it achieves the global convergence rate of $O(1/k^3)$ in terms of the functional residual. To the best of our knowledge, this is the first second-order method for this problem class attaining this rate while relying solely on one linear system solve per iteration (without solving auxiliary nonlinear regularized subproblems, such as cubic regularization, performing nonlinear parameter searches, or using dual extragradient corrections). Our method can be implemented in a Hessian-free way, using an inexact linear system solver, while preserving the fast global rate. We further extend our construction to arbitrary geometry through Bregman divergence, and to composite optimization problems.

VIALS: A Benchmark for Visual Interpretation of Artifacts in the Life Sciences cs.AI

In professional life sciences workflows, scientists routinely interpret visual artifacts (gel blots, microscopy images, plasmid maps, flow cytometry plots, molecular structures, ...) to inform research decisions. We introduce VIALS, a visual question-answering benchmark with 161 such interpretation tasks, spanning the types of artifacts examined throughout experimental workflows in the biotech industry (rather than polished figures from publications and textbooks). While frontier vision-language models can now fluently describe natural images, we find that they are unable to accurately interpret these scientific images, reflecting limitations in domain knowledge and domain-specific visual reasoning capabilities. In contrast, scientists with relevant domain expertise find these visual interpretation tasks straightforward. AI that cannot similarly interpret such images will have limited utility in professional life sciences workflows, where such artifacts are central to how scientists reason, communicate, and make decisions.

AI with Authority, from Application to Silicon cs.SE

For sixty years, machine verification has been a major cost overhead, affordable only for exceptional artifacts. Here we report that generative AI inverts this relationship: at AI speed, machine verification is not only economical but essential to productivity --- it is the incorruptible referee that lets one person safely direct autonomous machine work at scale. In five weeks, one researcher on consumer AI subscriptions directed a small fleet of AI agents from application code, through a verified compiler and executive, to a RISC-V processor taped out on a community silicon shuttle; no proof passed through human review, and no RTL was written by a human. The working discipline --- the Salt method --- rests on a proof kernel no hallucinated proof can pass: mathematical claims travel between agents as kernel-checked artifacts, and human attention is reserved for statements, designs, and rulings. Verification is stated link by link, from the Lean 4 kernel to SAT-checked equivalence at the silicon boundary. We publish the complete accounting: theorem provenance, a pre-registered token meter, floor-bounded human time, and an error ledger whose catch numbering runs to #256 --- a monotone counter over the mathematics campaign's append-only flags ledger, maintained 2026-07-07 to 2026-07-20 (one number, #79, was never assigned; later catches are recorded un-numbered) --- against zero incorrect proofs reaching the record.

PerturbRx: Learning Treatment-Conditioned Latent Transitions for Patient Drug Response Prediction q-bio.QM

Scarce data and tumor heterogeneity limit patient-level cancer treatment-response prediction. Existing approaches predict response from pretreatment molecular profiles and drug representations, without explicitly modeling the molecular changes expected under treatment. We propose PerturbRx, a treatment-conditioned representation learning framework that learns intervention-induced latent transitions and uses them as patient-drug response features. PerturbRx trains a drug- and dose-conditioned transition predictor from context-matched but cell-unpaired control and treated single-cell populations, then freezes and transfers the predictor to pretreatment patient profiles without requiring post-treatment measurements. The transition is combined with patient and drug representations to predict response. Across TCGA and patient-derived xenograft benchmarks, PerturbRx achieves the strongest aggregate predictive performance among the evaluated methods. These results support perturbation-pretrained latent transitions as useful representations for patient-level drug-response prediction.

Truthful Calibration Measures for Sequential Prediction cs.DS

Calibration requires probabilistic reports to be conditionally unbiased and reliably interpretable as probabilities. A calibration measure assigns numerical error to miscalibrated reports. Haghtalab et al. (2024) proposed an approximately truthful calibration measure for online prediction, leaving open whether exact truthfulness is compatible with completeness and soundness. We resolve this question negatively for sequential binary prediction: exact truthfulness is incompatible with completeness and soundness, even for independent outcomes. We then show that this impossibility is specific to exact truthfulness. We give two general reductions from a base calibration measure, producing additively and multiplicatively approximately truthful calibration measures, respectively. Applying the multiplicative reduction, for every $0 < \varepsilon < 1$ we construct a sound and complete calibration measure that is $(1+\exp(-T^{(1-\varepsilon)/2}/2))$-multiplicatively truthful. This improves the approximate-truthfulness guarantee of Haghtalab et al. (2024).

Asymmetric Capacity Allocation in Self-Refinement Pipelines cs.LG

Self-refinement, typically structured as generation, critique, and revision, is a widely adopted paradigm for improving LLM generation and serves as a core mechanism in many LLM agents. While the three stages involve different cognitive demands, most existing approaches conveniently treat the model size as an implementation detail rather than a subject of study, which may lead to a waste of resources. Little work has systematically examined how model size affects each stage or whether effective self-refinement requires equally capable models for generation, critique, and revision. We present the first stage-wise model size study of the self-refinement pipeline on 5 benchmarks from different domains using 6 model sizes of Qwen3 and 4 model sizes of Gemma 3. We conclude that larger generators and refiners generally improve the pipeline, whereas an undersized refiner can even harm performance. Second, performance is highly insensitive to the size of the critic, although including even a small critic consistently outperforms omitting critique altogether. Our findings demonstrate that model capacity should not be allocated uniformly across self-refinement pipelines. Instead, different stages exhibit distinct size scaling characteristics, providing practical guidance for designing more computationally efficient multi-stage language model systems.

BenchmarksFull tables
Artificial AnalysisIntelligence Index

Composite score across coding, math, and reasoning

#ModelScoretok/s$/1M
1Claude Opus 563.152$10.00
2Claude Fable 562.163$20.00
3GPT-5.6 Sol60.964$11.25
4Grok 4.660.956$3.00
5Kimi K359.734$6.00
SWE-rebench

Agentic coding on real-world software engineering tasks

#ModelScore
1AnthropicFable 5 [high]Model64.5%± 1.41%
2GrokGrok 4.5 [high]Model63.8%± 0.60%
3AnthropicOpus 5 [high]Model63.4%± 1.35%
4Z.aiGLM-5.2 [high]Model62.9%± 1.19%
5OpenAIGPT-5.6 Sol [medium]Model62.3%± 1.83%