The Inference Report

The agent framework market is consolidating around two competing visions. One camp treats agents as task executors that need infrastructure: InsForge and hermes-agent build backend scaffolding for multi-step workflows, while OpenRAG packages retrieval, parsing, and search into a single stack. The other treats agents as skill accumulators that grow through use, exemplified by vectorize-io/hindsight's learnable memory system and obra/superpowers' methodology-first approach. Both are solving a real problem, agents fail at scale without either solid operational foundations or adaptive learning, but they're solving it in opposite directions. The divergence matters because it determines whether the next layer of tooling goes into orchestration or into knowledge management.

Inference optimization is the other major theme running through this week's set. BitNet's 1-bit LLM framework and LiteRT's on-device ML runtime are both attacking the same constraint: getting capable models to run where they're needed without the cost of cloud calls. Google's A2UI and Anthropic's Claude plugin directory suggest the agent interface problem is also being solved at the platform level, not just the framework level. Meanwhile, the discovery repos reveal that the infrastructure beneath agents is still being built out, Weaviate's vector database with structured filtering, moltis' sandboxed tool runtime, and zenoh's data motion layer all address specific failure modes in distributed agentic systems. The pattern is clear: agents are moving from prototype to production, and developers are investing in the plumbing that makes that transition survivable.

Jack Ridley