The Inference Report

The trending set reveals two distinct gravitational pulls in developer tooling right now. On one side, there's a consolidation around conversational interfaces to LLMs, Onyx positions itself as a platform-agnostic chat layer that works across any model, while Oh My codeX and Prompts.Chat treat the LLM as a building block for agents and prompt management. These aren't solving new problems so much as making existing ones less painful: the friction of model switching, the tedium of prompt versioning, the need to coordinate multiple AI calls. Prompts.Chat's 157k stars reflects something less about the tool's technical merit and more about network effects in prompt sharing, it's a community collection that benefits from being the largest collection. Sherlock and OpenScreen occupy a different niche entirely, solving concrete, specific problems that don't require an LLM at all. Sherlock hunts usernames across social networks through pattern matching and HTTP requests. OpenScreen records and edits screen demos without subscriptions or watermarks, competing directly against Screen Studio on pure capability. Both solve problems that existed before LLMs and will exist after.

The discovery set shows where the real architectural work is happening. Microsoft's Presidio tackles PII detection and redaction across text and images with pluggable NLP pipelines, the kind of infrastructure that looks boring until you need it in production and realize how fragile ad-hoc solutions become. TimesFM from Google Research is a pretrained foundation model for time-series forecasting, applying the foundation model pattern to a domain where it hasn't dominated as thoroughly as language or vision. Genkit from Google is a framework for building AI-powered applications across multiple languages, which suggests a shift from "LLM wrapper" to "application runtime that happens to use LLMs." Roboflow's Inference and Dograh's voice agent platform indicate developers are pushing past chat interfaces into structured tasks, computer vision on edge devices, voice interaction. The pattern across these isn't about LLM capability or speed. It's about infrastructure: how to route data, how to manage state, how to keep private data private (Matthiasn's Lotti runs offline), how to deploy models reliably. Agents and RL are merging in the discovery set too, AgentsMeetRL curates that intersection, suggesting the next wave isn't better chat but systems that learn from interaction.

Jack Ridley