The Inference Report

May 17, 2026

Fragmentation is the story. The AI industry is sorting itself into winners and losers not by technological capability alone but by proximity to capital, regulatory favor, and distribution channels. OpenAI's partnership with Malta reveals the real playbook: lock in government relationships before competitors do, embed your product into civic infrastructure, and frame commercial distribution as a public good. Meanwhile, the infrastructure layer is splitting between monolithic all-in-one solutions like Bun and Superpowers that eliminate coordination overhead, and focused modular components like RAGLite that preserve flexibility for heterogeneous stacks. Both approaches are winning because they solve real friction, but they solve it for different developer profiles.

The fractures run deeper than product strategy. ArXiv's ban on outsourced language model research and bug bounty programs drowning in AI-generated noise both signal the same problem: automation floods any system with low friction and high noise tolerance, and the industry acknowledges this but won't solve it structurally. ByteDance and Kuaishou's lead in video generation exposes how Western AI dominance claims often reflect benchmark scores rather than production-grade systems that actually ship at scale. The CFTC's plan to deploy AI against insider trading in prediction markets reads as regulatory capture dressed as oversight. This is capital sorting itself toward massive concentration or narrow technical wins, away from the messy middle.

Research in human-computer interaction reveals what happens when this sorting completes: AI's apparent frictionlessness and sycophantic affirmation quietly shift human behavior and expectations in ways that feel effortless but carry hidden costs to judgment and epistemic integrity. Observability and knowledge management are emerging as first-class infrastructure concerns precisely because systems are becoming harder to inspect and understand as they grow more complex and interconnected. Developers are building private knowledge bases and on-device inference solutions to manage the cognitive overhead of working with multiple models. The unglamorous problems that compound at scale are now driving architectural decisions. The industry is not moving toward coherence. It is efficiently sorting capital, talent, and regulatory attention away from the messy middle.

Grant Calloway

AI LabsAll labs
From the WireAll feeds
Research Papers — FocusedAll papers
OnBodyAI - Why the Next AI Needs a Body cs.HC

The next AI will not win by knowing more about the world. It will win because it knows you: because it lives on your body, works for you and answers to you. That is what the term OnBodyAI stands for. It describes what happens when large language models are given a body, and what research and industry need to make that happen. This article is based on my talk in the Industrial Panel at UbiComp/ISWC 2026 in Shanghai (October 13, 2026) and follows its ten key statements.

What do AI chatbots do after users decline therapeutic redirection for suicide thoughts? cs.HC

Safety evaluations of artificial intelligence (AI) chatbots focus on therapeutic redirection, statements urging users who disclose suicide thoughts to seek help from crisis lines or other human sources. How chatbots respond when users refuse that help is unclear. This cross-sectional audit ran 2,550 scripted multi-turn conversations in September 2026. Each conversation opened with a disclosure of suicide thoughts at 3 levels of severity, answered every referral with up to 10 standardized refusals and then changed the subject. All 2,550 conversations included a referral, 2,542 (99.7%) in the first reply. However, mean persistence ranged from 0.01 to 9.94 refusals across chatbots. Newer versions of all 5 chatbots tested in 2 versions persisted less than older versions (each P < .05). Compared with passive ideation, a disclosure of intent was associated with 1.26 more refusals sustained (95% CI, 1.06-1.46) and higher odds of renewed referral after the topic change (odds ratio, 10.07; 95% CI, 7.43-13.66). Chatbots that persisted less in referral more often presented themselves as the user's support (Spearman rho = -0.80; P < .001). Of 8,818 refusal-phase replies urging outside help, only 173 (2.0%) asked about barriers to seeking it. Chatbots uniformly recommended outside help after disclosures of suicide thoughts but varied widely in persisting in those recommendations, with newer versions persisting less. Safety evaluations should test what happens after a crisis referral is refused, not only whether one is offered.

Following Delegation Traces to Study Human Oversight in Interaction with Multimodal AI cs.HC

As multimodal AI systems are becoming our everyday thought partners, the shift towards studying how users transfer control to these systems becomes especially urgent. In this paper, we report an overview of findings from an online study with n=56 AI-literate participants and 392 Video Question Answering trials. After inspecting each multimodal AI response, participants rated task-specific trust and selected one of three intended policies for an analogous future task: reject AI assistance, use it with verification, or delegate fully to the AI. Within participants, post-interaction trust strongly predicted the user's delegation choices (OR=14.97, 95% CI [7.14,31.37]). Trust and delegation nevertheless remained distinct. Verification was the dominant policy and often co-occurred with high trust. Delegation also increased with task order after adjustment for momentary and participant-level trust (OR=1.22, 95% CI [1.02,1.45]), indicating an experience-related component that self-reported trust ratings alone could not capture. These findings motivate a shift from intended delegation to a non-intrusive observation of how delegation is enacted in collaborative human-AI tasks. We therefore conclude by proposing enacted delegation traces as a methodological framework for future studies of multimodal and agentic AI use.

Task-State Governance: A Virtual Machine for Verifiable and Recoverable GUI Agent Execution cs.HC

Graphical user interface (GUI) agents now execute workflows across real applications, but when execution drifts, users must determine what changed, what should have remained untouched, and what can still be recovered. Plan-centric interfaces leave users supervising procedure, but users live with changes to application state. We introduce task-state governance, an interaction paradigm that represents a task as an editable contract linking desired outcomes, facts that must be preserved, and observed application state. We instantiate this paradigm in TaskVM, which projects live state into editable task controls, executes user edits as genuine interface actions, verifies outcomes through independent observation, and marks each action as reversible, compensatable, or irreversible. Across an automated contract benchmark, a controlled user study, and an open live-model exploration, the paradigm strengthened contract compliance and users' recovery control and trust, while execution outcomes were held constant. Task contracts open a design space for governable agent interfaces.

What and Where in Identifying and Quantifying Conventions in Humans and AI Agents cs.HC

Cooperative success rests in part on implicit communication. Partners converge on conventions: regularities that give an action meaning beyond its direct effect. Yet conventions are usually observed only indirectly: a score drop when partners are swapped shows that a convention existed, but not what it was or where it was used. In the card game Hanabi, the exact probability that a play fails can be computed from the hints received and the cards in view. Players who use conventions succeed more often than this prediction allows; the resulting margin is the convention gap. Off-belief learning provides a hierarchy of agents (levels 1-5) whose first level uses no conventions by construction and whose later levels use progressively more. For humans and these agents on one scale, we ask what the conventions are and where they are used. On what: in two independently collected corpora of human games, humans show a larger gap than every agent level. On the conventions both populations use, however, humans do no better than level 2. The human advantage is concentrated in a situation that no agent level learns to use: hints that point to a card other than the ones they touch. On where: conventions depend on what the player already knows about the card, not on how recently it was hinted, and they are used most in the mid-game; humans use them earlier and later in the game than the agents. In interventions on the agents, blocking a single high-gap situation costs little, because meaning reroutes to other situations, but blocking several together costs far more than the sum of the single costs. Thus, the convention gap turns conventions from an inferred cause of score loss into a measurable, comparable, and causally testable object.

Intent Graph: Navigating the Analytical Reasoning Space for Exploratory Data Analysis cs.HC

Exploratory data analysis (EDA) is rarely open-ended in practice: analysts work from high-level domain questions toward the concrete analyses that can answer them, prioritizing directions with domain knowledge and prior hypotheses. Large language models (LLMs) can supply such knowledge, but their responses are unstructured, leaving analysts no way to see what has been explored, what is missing, or why one direction was chosen over another. We present DAG-EDA, a system that lets analysts and an LLM co-navigate the space of possible analyses through two linked structures. An intent graph, governed by a grammar of analytical intent, decomposes an ambiguous natural-language question into progressively concrete analysis tasks, keeping alternative framings open and letting analysts branch, backtrack, and compare paths. A multi-layered knowledge graph externalizes the LLM's domain knowledge, linking domain concepts to the dataset variables that can measure them, so analysts can inspect and contest how their question is grounded in the data. Both graphs are constructed from only the dataset and the analyst's question, and the analyses the analyst reaches are rendered as interactive dashboards. We illustrate the system through a usage scenario and describe a user study design for examining whether the system scaffold analysts' reasoning and navigation.

BenchmarksFull tables
Artificial AnalysisIntelligence Index

Composite score across coding, math, and reasoning

#ModelScoretok/s$/1M
1GPT-5.560.275$11.25
2Claude Opus 4.757.351$10.94
3Gemini 3.1 Pro Preview57.2131$4.50
4GPT-5.456.882$5.63
5Kimi K2.653.944$1.71
SWE-rebench

Agentic coding on real-world software engineering tasks

#ModelScore
1Claude Opus 4.665.3%
2gpt-5.2-2025-12-11-medium64.4%
3GLM-562.8%
4Junie62.8%
5gpt-5.4-2026-03-05-medium62.8%