The Inference Report

The AI agent infrastructure layer is crystallizing around two distinct patterns. First, the orchestration and deployment tier: repos like deer-flow and ruflo are building runtime environments that handle multi-agent coordination, memory management, sandboxing, and tool integration. These aren't just wrappers around LLM APIs. They're solving the operational problem of keeping agents running reliably over hours-long tasks, managing state across multiple tools, and coordinating between specialized sub-agents. The second pattern is domain-specific agents built on top of that infrastructure. dexter handles financial deep research, local-deep-research builds a retrieval system optimized for factual accuracy across multiple sources, and InsForge provides the backend layer (Postgres, auth, storage, compute) that agents need to persist data and execute code. What's notable is the assumption baked into all of these: agents need to be stateful, long-running, and tightly integrated with external tools and knowledge sources. The single-turn LLM call is no longer the unit of work.

Alongside this, a different category of tools is gaining traction: infrastructure that makes building and deploying agents cheaper or more accessible. free-llm-api-resources aggregates inference endpoints. TabPFN brings foundation models to tabular data, solving a specific prediction problem without requiring fine-tuning. Scrapling and docuseal handle data acquisition and document workflows that agents need but don't want to build themselves. And the discovery-tier repos reveal what's upstream: fine-tuning frameworks like unsloth-buddy, synthetic data generation from DataDesigner, and the realization that agent builders need to control their own data pipelines rather than relying entirely on closed APIs. The trend isn't just that agents exist. It's that the entire stack around them is being rebuilt to support longer horizons, statefulness, and cost control. The framework wars are moving from "which LLM" to "which agent runtime and which set of specialized tools."

Jack Ridley