The Inference Report

October 9, 2026

AI infrastructure is consolidating around control points rather than capability, and the companies racing to own those chokepoints are signaling where the real money will be. Google's universal agent accessible through a single prompt and API across Gemini, Microsoft's expansion into local Windows file systems via Copilot, and Amazon's open-source safety sandbox for constraining agent behavior each represent a different bet on which layer of the stack becomes defensible. AWS's position is the clearest: by owning the guardrails that make agents deployable at enterprise scale rather than the agents themselves, the company has chosen the more defensible ground. Nvidia is playing a different game entirely, bundling safety into the hardware and software stack for robotics to compete for the physical world rather than the software layer. The revenue picture underneath these launches tells the real story. OpenAI's actual annualized revenue is $50 billion, not $70 billion as widely reported, and Firmus, an Nvidia-backed data center operator, pulled its $5 billion IPO citing market volatility, a signal that investors are growing skeptical of AI infrastructure valuations divorced from actual returns. Arena, the AI leaderboard, raised $200 million and nearly doubled its valuation to $3.1 billion in ten months by shifting focus to measuring alignment and deception rather than raw model performance. Capital is chasing measurement infrastructure because the capability story has become commoditized.

The research and development landscape is bifurcating along similar lines. OpenAI is showcasing cost efficiency and workflow automation through Oracle, LegalOn, and creative tools, messaging that their models are already cutting real spending and accelerating work. Anthropic is broadcasting commitment to American science, cyber defense, and open-source vulnerability scanning, positioning itself as the trustworthy alternative rather than the efficiency leader. Nvidia and AMD are locked in a technical specification war over accelerator performance and cost per unit, with Nvidia framing its $1 billion commitment as positioning for "super intelligence research" to sidestep commodity chip competition. The labs are no longer competing on the same axis. One track rewards builders shipping products that reduce customer costs, another rewards infrastructure vendors proving performance per dollar, and a third rewards labs credibly claiming alignment with public goods and national priorities.

The research papers themselves reveal where the actual technical work is happening. Safety-focused work is tackling verification in embodied and autonomous systems through filtering, monitoring, and assurance frameworks, while parallel efforts address representation learning and prediction in high-dimensional domains. The methodological emphasis across both streams is on decoupling objectives: safety from task performance, representation learning from task-specific fine-tuning, measurement from leaderboard position. At the benchmark level, SWE-rebench shows convergence at the top tier with AnthropicFable 5 holding 64.5% and the next five models clustered within 2.2 percentage points, suggesting either genuine capability convergence or plateau in the test's discrimination power. Artificial Analysis reveals churn in the middle ranks across its 470-entry leaderboard, but those shifts often fall within measurement noise, raising questions about whether the fluidity represents meaningful performance changes or experimental variance.

The GitHub ecosystem is splitting between agent infrastructure and production-ready tooling. Haystack, Claude-mem, and Xinference are solving concrete problems that emerge once you move beyond single-turn interactions: orchestration, context persistence across sessions, and model abstraction. Microsoft's governance toolkit addresses what becomes urgent once agents run unsupervised. The second wave involves tools built explicitly for agent-centric workflows: diagram design for Claude Code integration, skills repositories standardizing on agent patterns, and domain-specific applications like reverse-engineering and platform porting. These repositories signal the transition from agents as chatbots to agents as production systems, which means persistence, governance, model abstraction, and specialized stacks for actual work. The winners in this cycle will be the companies that own the distribution channels, set the standards everyone else builds around, and solve the friction points that emerge at scale.

Grant Calloway

AI LabsAll labs
From the WireAll feeds
Research PapersAll papers
CSF: Contextual Safety Filtering for Motion Generators cs.RO

Text-conditioned motion generators produce trackable whole-body motion, but they have no notion of scene-dependent safety: the same action may target an object or a person. Existing safeguards either inspect the prompt, require labeled motion data, or enforce geometric constraints; therefore, they do not directly account for how scene context changes a motion's meaning. We introduce contextual safety filtering (CSF), a training-free filter that grounds natural-language safety rules in safe and unsafe reference trajectories produced by the generator. For each active rule, safe and unsafe reference trajectories define an affine safety value that a safe reference tracking CBF-QP enforces. Across four pretrained generators with different architectures, CSF activates the intended rules in all explicit and scene-triggered unsafe cases and reduces the danger-event rate by up to 90%, while preserving 88-100% of benign motions. We demonstrate the complete system on a real-world Unitree G1, where it successfully prevents unsafe motions in a variety of scenarios, including interactions with humans and objects.

On the estimation and validity of AI time horizons---a statistical look at the METR plot cs.AI

METR's 50\% time horizon measures the human completion time of software tasks that an AI solves with 50\% probability, allowing AI capabilities to be expressed in interpretable units. On 228 tasks and 26 AIs, we recompute the time horizons using splines and item-response theory to relax the assumption that the AI difficulty of a task depends linearly on the log of human time. Our fitted spline can be interpreted as a function that \emph{converts} human time to AI difficulty; it is nearly flat in a region from 2--30 min but close to linear elsewhere. Hence, a time-horizon jump from 3 min to 30 min is much easier than one from 30 min to 5 hours despite the same multiplier of $10 \times$. Overall, we contribute time-horizon point estimates that perform better under a cross-validated suite of proper scoring rules, as well as diagnostic plots for assessing time horizons' construct validity. We suggest that time horizons be interpreted together with the diagnostic plots, especially as new time-horizon-based benchmarks are proposed or existing ones grow to include longer tasks.

A Balanced Data Diet: Addressing the Exploration Bottleneck in Mega-Scale RL for Robot Control cs.RO

General-purpose robots must perform a wide range of tasks from agile locomotion to dexterous manipulation. While sim-to-real reinforcement learning (RL) has proven to be a useful tool for this goal, current RL pipelines depend on engineering-heavy, per-task structural priors such as shaped rewards and demonstrations. Recent work has shown that diverse simulator resets, combined with massively parallel simulation, can alleviate much of this engineering burden on several manipulation problems. However, we find that naively scaling this paradigm to more precise or dynamic problems remains non-trivial. While simulator resets can help with exploration, uniformly sampling over this distribution wastes a growing fraction of learning experience on task configurations the policy has already mastered or cannot yet attempt. This makes it challenging to see the expected benefits of scaling parallel environments for RL, since much of the learning signal in a batch is wasted during learning. To mitigate this, we introduce Success Guided Sampling (SGS), a simple adaptive sampler that concentrates RL training on task configurations around the frontier of the policy's capabilities. Doing so allows large-scale simulated RL to make the most out of the experience in a batch, enabling much more effective scaling to large-scale parallel simulation. Across experiments using up to $2^{20}$ (over one million) parallel environments, SGS enables RL to solve challenging multi-terrain quadruped locomotion and contact-rich assembly tasks that prior methods fail to solve. Finally, we distill the learned manipulation policies into RGB-based policies and demonstrate zero-shot transfer to several challenging assembly tasks on real hardware. Project website: https://sgs-rl.github.io/.

From Reactive Containment to Proactive Assurance: Lessons from OpenAI, Anthropic, and Google Agent Security Incidents cs.CR

In 2026, cybersecurity evaluations involving OpenAI, Anthropic, and Google agents reached real systems outside their authorized test scope. The paths were different. OpenAI agents exploited research infrastructure, coordinated across runs, and compromised parts of Hugging Face's production environment. Anthropic reported cases in which a misconfigured third-party environment exposed real systems to agents pursuing simulated cyber tasks. In a separately reported evaluation, Google's Gemini accessed three real organizations through an unintended internet route; Google stated that the model stopped in all three instances. Taken together, the cases show why an evaluation cannot rely on an assumed boundary. That boundary must be verified while the agent is operating. This comparative instrumental case study develops a Proactive Agent Security Assurance Cycle (PASAC) and a five-layer Boundary Assurance Stack. The framework combines risk-tiered task design, executable scope contracts, pre-run validation, least-capability access, independent egress enforcement, credential restrictions, cross-run monitoring, automatic stop conditions, and evidence-based reauthorization. A leading-indicator model, nine design propositions, and seven falsifiable hypotheses turn these lessons into a testable research program. Because the public Gemini record is limited to attributed statements and journalism, its detailed causal mechanism remains provisional. The central conclusion is straightforward: proactive agent security requires continuous assurance across the full execution system, not confidence in any single sandbox or safeguard.

BrickBench: Evaluating Agentic Brick Design cs.AI

We propose BrickBench, a benchmark for agentic text-conditioned LEGO-set design. Given a prompt, an agent is tasked with producing an assembly that not only satisfies semantic and design criteria, but that can also be physically built. To do so, it must select parts from a discrete library and reason jointly about local and global constraints. We score validity, alignment, and design across three settings that vary in scale and part availability. We provide BrickAgent, an environment for coding agents to construct, inspect, and validate their designs. We find that leading agents largely satisfy verifiable physical and semantic requirements, but fall short of human designs. We release our benchmark and environment at http://www.brickben.ch

One Block, Multiple Depths: Recurrent Vision Transformers with Depth-Programmed Experts cs.CV

In this work, we show that a single Transformer block, applied recurrently, can match the accuracy of a full-depth vision encoder at comparable inference FLOPs without intermediate feature distillation. reViT restores depth-specific transformations by representing the FFN at each recurrent depth as a convex combination of a small shared expert bank. A continuous normalized-depth coordinate programs this mixture, defining a resampleable trajectory through FFN parameter space. We evaluate this design in two regimes: supervised ImageNet-1k training and distillation from a DINOv2 teacher. Across both regimes, controlled adaptations identify weight-space merging as the strongest tested MoE family at a matching one-FFN budget, ahead of the token-dispatch and output-mixture alternatives. Trained from scratch, reViT-B/16 attains DeiT III accuracy with about 70\% fewer stored parameters. An 8-experts model distilled using only the teacher's output features retains nearly all of its DINOv2 teacher's linear-probe accuracy and transfers across classification, segmentation, and depth prediction. Elastic-depth training allows one checkpoint (trained model) to operate at multiple tested depths by resampling the same normalized coordinate interval. For fixed-depth deployment, the recurrent block can be materialized as a conventional dense graph, removing online routing and merging without changing the one-FFN-per-depth compute but expanding deployment storage.

BenchmarksFull tables
Artificial AnalysisIntelligence Index

Composite score across coding, math, and reasoning

#ModelScoretok/s$/1M
1Claude Opus 5.557.697$8.00
2Claude Sonnet 5.556137$4.00
3Claude Fable 5.153.472$20.00
4GPT-6 Astra52.747$20.00
5Gemini 4 Argon52.60$4.00
SWE-rebench

Agentic coding on real-world software engineering tasks

#ModelScore
1AnthropicFable 5 [high]Model64.5%± 1.41%
2GrokGrok 4.5 [high]Model63.8%± 0.60%
3AnthropicOpus 5 [high]Model63.4%± 1.35%
4Z.aiGLM-5.2 [high]Model62.9%± 1.19%
5OpenAIGPT-5.6 Sol [medium]Model62.3%± 1.83%