Behind the Curtain: Intelligence explosion (axios.com)

Anthropic , the AI lab whose identity is wrapped around warning the world about AI risk, is claiming "early signs" of AI not just coding its own products but building itself. Why it matters: Anthropic co-founder Jack Clark predicted this week that there's a 60%+ chance of an AI model fully training its successor by the end of 2028. "What I'm looking at is a technological trend where, if anything, the speed will accelerate further," Clark told us. In the new research agenda for The Anthropic Institute — first shared with us, and out Thursday — the company says it's seeing signs of "AI contributing to speeding up the research and development of AI itself," a process known as recursive self-improvement. And Anthropic researchers think the world should know. "My prediction is by the end of 2028, it's more likely than not that we have an AI system where you would be able to say to it: 'Make a better version of yourself.' And it just goes off and does that completely autonomously," Clark, who heads the institute, told us from Anthropic headquarters in San Francisco. "It's always been the case that humans outside the technology need to come up with the ideas that they then put back into it. What happens if we have a technology that can generate ideas within itself for how to improve itself? That's a new concept." The five-page document warns of a possible "intelligence explosion" — long a theoretical term confined to AI safety circles. Now it's in writing, in an official Anthropic document. Clark told us an intelligence explosion is when AI systems suddenly start improving at blinding speed. Lots of bad things can happen (cyber meltdowns and biological attacks). And lots of good: "What do you do with a tremendous amount of growth or a tremendous amount of abundance in many, many different fields of science all at once?" he asked. "Today's institutions have very, very narrow pipes through which you push new drug candidates. How do you massively broaden the size of those pipes in advance of this abundance?" What's new: The Anthropic Institute is part research arm, part early-warning system, with an agenda built alongside Anthropic's Long-Term Benefit Trust . (Clark also has the title "head of public benefit.") The research agenda focuses on four buckets: Economic diffusion: jobs, productivity, who captures the gains. Threats and resilience: cyber, bio, surveillance. AI systems in the wild: agents, governance. AI-driven R&D: the recursive self-improvement question. The promise: Anthropic is committing to publishing more "detailed information about how our work at Anthropic has sped up as a result of new AI tools, and ideas about the implications of potential recursive self-improvement of AI systems." Translation: A frontier lab is on the record promising to tell the public when the machine starts building itself. If AI is building itself, will we need AI companies? "We and the other companies are going to be taking this technology and trying to get it to d