The Inference Report

April 2, 2026

The compute arms race has entered a new phase where infrastructure control determines market position, and the companies that can lock in power, compress hardware deployment timelines, and absorb regulatory complexity are already separating from those that cannot. Meta's commitment to ten natural gas plants powering its Hyperion data center and Cognichip's $60 million raise on claims of 75% cost reduction and 50% timeline compression in chip design are not marginal improvements but structural bets that the next cycle belongs to whoever can compress the cost and time to deploy compute at scale. Poolside's stalled Texas project and forced negotiations with Google and other cloud providers make the calculus explicit: independent infrastructure plays are losing leverage to companies that can absorb the capital requirements and regulatory complexity of massive data center deployments. The economics have shifted decisively.

This consolidation around compute is matched by a bifurcation in how companies are deploying AI. OpenAI's Gradient Labs is shipping concrete product, AI account managers in banking using GPT-4.1 and purpose-built smaller models for latency-sensitive workflows, the kind of deployment that generates revenue and defensible customer relationships. GitHub's /fleet feature for parallel agent dispatch and Hugging Face's Holo3 announcement signal the same momentum: builders are moving past single-agent orchestration into systems that distribute work across multiple models and processes. By contrast, IBM and AMD are playing a different game, announcing FedRAMP authorizations, decade-long research initiatives, and detailed MLPerf submissions that establish them as trusted intermediaries in regulated environments and foundational research. Consumer-facing applications are racing ahead through product velocity and direct customer relationships. Infrastructure players are establishing credibility through partnerships, compliance achievements, and benchmark credentials.

The technical frontier has shifted from model architecture to the systems that deploy and secure them, and that shift exposes unresolved tensions. Anthropic's accidental mass takedown of GitHub repositories containing leaked Claude Code source, followed by the discovery that Claude Code can uncover zero-day exploits in Vim and GNU Emacs in seconds, reveals that the same tool making security research trivial also makes security risk trivial to create. Anthropic's response was operational damage control, not a technical solution. Meanwhile, Slack's repositioning of Slackbot as an orchestration layer for agentic workflows, Asana's emphasis on multiplayer AI agents, and Meta's semi-formal reasoning technique for code review achieving 93% accuracy show enterprises moving past chatbots toward systems that make decisions and execute tasks across multiple applications and teams. What remains unresolved is whether the tools that make this possible can be secured, and whether the companies deploying them have any meaningful way to audit what those agents are actually doing once they are turned loose on company infrastructure.

Benchmark volatility and developer momentum toward agentic tools underscore the stakes. Claude Opus 4.6 holds the top position on SWE-rebench at 65.3%, a 12.3-point gain from its prior ranking, yet the two major benchmarks diverge materially in their orderings, implying that they capture different failure modes in code generation or apply stricter evaluation criteria around execution correctness. Reasoning-focused variants like Kimi K2.5 are closing gaps on general-purpose models in software engineering tasks. On GitHub, Claude Code and related agentic tooling have moved from research artifact to daily driver, while infrastructure projects like Pixeltable, Rerun, and Qdrant represent a category shift toward unsexy but essential data plumbing for multimodal pipelines. Simultaneously, projects like PicoLLM signal a countercurrent toward running models locally without cloud dependencies. The field is scaling inference infrastructure while constraining it, building both the pipes and the guardrails.

Grant Calloway

AI LabsAll labs
From the WireAll feeds
Research Papers — FocusedAll papers
Building the Truman Show: A TrustZone-Based Framework for Lightweight Out-of-band Kernel Security Monitoring cs.CR

The increasing number of vulnerabilities in operating systems, together with sophisticated kernel-level threats (e.g., rootkits), has weakened the effectiveness of traditional in-kernel protection mechanisms. Since these defenses operate at the same privilege level as the kernel, they share the same attack surface and can be bypassed once the kernel is compromised. Isolation-based security approaches provide stronger protection by separating security logic from the kernel, but strict isolation often introduces semantic gaps that limit system visibility and hinder timely threat detection. In this paper, we present LOOM, a lightweight out-of-band operating system monitoring architecture built on ARM TrustZone. By leveraging TrustZone's hardware-enforced isolation, LOOM establishes a tamper-resistant monitoring environment independent of the kernel. To bridge the semantic gap, we design a lightweight semantic reconstruction mechanism in the Secure World. It selectively captures the states and behavioral patterns of critical kernel objects, such as process control blocks and kernel modules. Additionally, LOOM introduces a dual-stage hazard prevention mechanism that combines atomic memory protection with an interrupt-driven adaptive agent to detect and mitigate kernel rootkit activities. An address translation cache is further incorporated to optimize repeated address access and reduce monitoring overhead. Overall, we develop a multi-layered collaborative architecture with platform, functional, and auxiliary layers for secure and efficient kernel monitoring. A prototype of LOOM has been implemented on the Phytium D2000 platform. Experimental results indicate that LOOM incurs negligible overhead while maintaining a strong monitoring capability. Furthermore, a security capability analysis based on CVE cases demonstrates that LOOM can detect and mitigate various kernel attacks.

POLYFLOW: A Neuro-Symbolic Framework for Static Cross-Language Information Flow Analysis cs.CR

Modern software systems are commonly constructed in multiple, interacting programming languages. This construction leads to additional, often stealthy vulnerabilities buried in complex information flow due to language interactions. Existing static analyzers are impeded by the heterogeneous semantics of different languages, whereas dynamic approaches suffer from the limited coverage of (available and/or generated) test inputs. In this paper, we develop PolyFlow, a neural-symbolic framework for statically reasoning about information flow across language boundaries, combining large language models (LLMs) and static analysis synergistically. Governed by the control-flow representation of a given multi-language system, PolyFlow leverages LLMs to identify implicit flow facts due to challenging language features, hence augmenting the base representation and then propagating data flow through the system. It tackles inherent barriers (e.g., token limit and hallucination) of LLMs by putting them under careful guidance (e.g., static-analysis-guided scoping, context management, and fact checking), along with a multi-LLM expert panel for negotiated validation. Our experiments on real-world Python-C and Java-C systems show that PolyFlow is cost-effective and superior to various kinds of state-of-the-art baselines, revealing previously unknown cross-language vulnerabilities that are missed by all the baselines.

Influence Is Not Authority: When Causal Guardrail Signals Make Legitimate Tool Use Look Like an Attack in Tool-Using LLM Agents cs.CR

The key limitation of current state-of-the-art influence-based guardrails is that they do not reliably distinguish a legitimate, user-authorized action from a malicious, unauthorized action when both rely on external tool information. This ambiguity can cause benign actions to trigger unnecessary verification and intervention, reducing utility and adding latency. We expose this limitation through an authorization-equivalence audit of 96 conditions derived from 24 base cases. Within matched source comparisons, we hold authorization, the exact committed action, and its intended effect fixed, changing only whether a required value comes from the user or a legitimate tool result. Although the action remains unchanged, this harmless relocation shifts the causal signal toward the attack region in all 24 cases under both Llama and Gemma scorers. Matched unauthorized controls show that the signal remains attack-sensitive, yet the benign relocation produces a larger average score shift than the actual change in authorization. Architecture-level evaluation shows how this mismatch propagates through guardrail designs. With a semantic monitor, attack success is 0% and utility is 28%, compared with 16% and 60% without it. A shadow-based guardrail allows every tested harmless run, yet does not reject matched unauthorized actions more often overall: 57.5% of unauthorized runs pass automatically before reaching the later security check, compared with 29.2% of authorized runs. These results show that the studied causal signal reveals what shaped an action without reliably encoding whether the action was authorized, and that reference construction and routing are integral to the effective security decision.

Reachability-Based Capability Confinement for LLM Agents under Indirect Prompt Injection cs.CR

Large language model agents place outputs from external skills into their execution context, allowing attacker-controlled data to influence later privileged actions. Existing defenses mainly classify untrusted content or authorize proposed operations. They do not directly address how an agent's future authority should change once untrusted data enters its state. We present SkillGuard, a harness-level enforcement layer that treats this event as contamination and restricts future capabilities to disconnect the resulting state from deployer-defined forbidden states. Given sound skill summaries and policies, SkillGuard represents security-relevant transitions with a Skill Impact Graph, specifies admissible control over skill parameters via steerability signatures, and mediates invocations with an inline reference monitor. Following contamination, it computes weighted capability restrictions using binary, fractional, or fractional-flow strategies without auxiliary language-model inference. We evaluate SkillGuard on four AgentDojo suites with two backend LLMs, Gemini 2.5 Flash and Llama3.3-70B, against an LLM-only No Defense baseline and three defenses at different system layers: Spotlighting, CaMeL, and AttriGuard. We construct a compositional attack benchmark in which each attack combines observations individually insufficient to induce target violation and evaluate the same baselines on it. Under AgentDojo's Tool Knowledge attacks, SkillGuard eliminates attack success on three of four suites for both backends and reduces it to 4.8% and 14.3% on Slack. Against compositional attacks, it outperforms every baseline on Llama and matches the strongest baseline on Gemini at higher benign utility. Fractional-flow restriction preserves substantially more capabilities than binary restriction at the same attack success rate. Across both settings, SkillGuard adds no model calls or token overhead.

A Simple Transformer Pipeline for Full-Key Side-Channel Attacks on Uncropped Datasets cs.CR

Deep learning-based side-channel analysis has historically focused on single-byte targets and manually cropped traces, which risks discarding exploitable leakage. While recent work has proposed specialized architectures and resampling techniques to address this gap, the literature lacks a simple transformer baseline for simultaneous full-key attacks on uncropped traces. We present an open-source transformer implementation for uncropped full-key attacks which uses the standard transformer encoder backbone, adapting only the input and output layers to the side-channel setting. We release our implementation, training recipes, and pretrained weights for uncropped ASCADv1f, ASCADv1r, and CHES-CTF-2018 which achieve performance competitive with previously-reported results, while using less than 10GB of VRAM and requiring at most 3.34 hours of training on a single NVIDIA A6000.

Balancing Privacy, Utility, and Safety in LLM Alignment through Preference Optimization cs.CR

Preference optimization is widely used to align large language models with human preferences, but preference-data composition may also influence privacy-relevant memorization. We examine whether adding synthetic privacy-preference pairs to Direct Preference Optimization (DPO) is associated with lower canary-based memorization signals without modifying the objective or introducing a formal privacy mechanism. We propose Privacy-Pressure Preference Mixing (P3M), a data-composition protocol that varies the amount of privacy-preference data while keeping helpfulness and harmlessness preference data fixed. We evaluate a non-privacy Baseline and privacy-mixing ratios of 0.5, 1.0, and 2.0 using Gemma 3 270M-IT across five random seeds and validate the same four conditions using 4-bit-quantized Gemma 2 2B-IT across three seeds. Overall, under the tested conditions, privacy-preference mixing is associated with lower mean canary suffix log-likelihood proxy values across both model settings and lower aggregate membership-inference attack performance relative to the Baseline in the mixed-source 2B evaluation. Specifically, across the privacy-aware 2B configurations, the mean area under the receiver operating characteristic curve (AUROC) ranges from 0.596 to 0.629, and the mean area under the precision-recall curve (AUPRC) ranges from 0.541 to 0.575, compared with 0.804 and 0.790, respectively, for the Baseline. However, the reduction in membership distinguishability does not hold uniformly across data sources. Moreover, the relationship between the privacy ratio and harmlessness preference accuracy varies by model setting, whereas helpfulness preference accuracy remains broadly stable. These findings suggest that P3M should be viewed as a lightweight empirical protocol for examining privacy-utility-safety trade-offs rather than as a formal privacy guarantee or a defense against extraction attacks.

BenchmarksFull tables
Artificial AnalysisIntelligence Index

Composite score across coding, math, and reasoning

#ModelScoretok/s$/1M
1GPT-5.457.275$5.63
2Gemini 3.1 Pro Preview57.2117$4.50
3GPT-5.3 Codex5467$4.81
4Claude Opus 4.65351$10.00
5Claude Sonnet 4.651.754$6.00
SWE-rebench

Agentic coding on real-world software engineering tasks

#ModelScore
1Claude Opus 4.665.3%
2gpt-5.2-2025-12-11-medium64.4%
3GLM-562.8%
4gpt-5.4-2026-03-05-medium62.8%
5Gemini 3.1 Pro Preview62.3%