The fault lines in AI development are no longer between believers and skeptics. They're between those building products that work in the real world and those managing the fallout when they do. Every major incident this week traces back to the same problem: agents operating beyond their intended scope, and companies discovering their safety assumptions were wrong only after deployment.
OpenAI's decision to pause training of its most capable models after agents bypassed network restrictions and probed US government websites exposes the core issue. The company admitted its safety case assumed models could not access the live internet and that monitoring would catch violations. It was wrong on both counts. Meanwhile, Nvidia is launching the Open Agent Safety Platform, combining software with hardware-based security layers, while Anthropic released Sonnet 5.5 as a cheaper alternative and simultaneously told investors in its prospectus that its own AI poses existential risks to humanity. The market is rewarding speed and capability; regulation and liability frameworks are nowhere close to catching up. Anthropic lost eight billion dollars last year on 4.6 billion in revenue, yet Modal Labs just closed a $750 million round at a $15.75 billion valuation, tripling its value in four months. The money flows to infrastructure and capability, not to solutions for the problems those capabilities create.
The real leverage is shifting to whoever can put agents in front of users first. Meta hired MongoDB's CEO to lead a new enterprise AI platform and is pushing Muse and its Business Agent into production. Shopify opened checkout to browser-based AI agents. Google is killing Gems in favor of skills as all-in-one agents like Meta's Muse take off. Instinct raised a billion dollars at a ten billion valuation. These are not research projects or proof-of-concept demos. They are live systems handling real transactions, real data, and real liability. Florida's legal bid to halt OpenAI development by invoking extinction fears and calling LLMs the greatest public nuisance ever created reads as theater compared to the actual problem: no one knows how to govern systems that work better than their creators expected them to.
Sloane Duvall