Dario Amodei, chief executive of the giant artificial intelligence company Anthropic, refers to his fellow AI moguls as “chaotically oriented actors.” He has repeatedly called for the US government to regulate the industry; his peers, and the Trump administration, have opposed any constraints whatsoever. “I think we should be thinking about regulating AI the way you regulate cars and aeroplanes,” Amodei says. “Everyone realises they have enormous economic value, but they need to be built carefully. If they aren’t built right, they can kill you.”
We must hope Amodei’s carefully cultivated image as the conscience of the industry is accurate, because earlier this month his company — best known for its Claude AI models — revealed a new AI product, Mythos Preview, that’s widely seen as a step change in AI capabilities. Challenged by its designers to break out of a “cyber cage,” this model not only made its escape but also demonstrated a remarkable capacity to identify exploitable flaws in widely used software. “Mythos Preview has already found thousands of high-severity vulnerabilities,” said Anthropic, “including some in every major operating system and web browser.”
The model can identify tiny flaws in the systems that operate digital communications, electricity infrastructure, online banking and other vital services, and link them together to create major vulnerabilities open to exploitation. “Given the rate of AI progress,” the company added, “it will not be long before such capabilities proliferate, potentially beyond actors who are committed to deploying them safely.” In the wrong hands, Mythos could be used to bring essential infrastructure and services to a grinding halt.
“The ability to defend against cyberattacks is integral to the basic functioning of society,” observes Atlantic technology writer Matteo Wong. Mythos’s near-inevitable abuse would make that task much, much harder.
To underline Mythos’s alarming potential, Anthropic has withheld the model from general release and contacted forty big companies whose digital infrastructure could be threatened by its talent for spotting vulnerabilities. These companies — Amazon Web Services, Apple, Broadcom, Google, JPMorganChase and NVIDIA among them — have been given access to Mythos to scan and secure their own software as part of Project Glasswing, the company’s attempt to moderate the risks posed by its own product.
The significance of the model’s release didn’t elude governments and regulators around the world. The European Central Bank, for instance, began questioning banks about their digital safeguards. Canadian finance minister François-Philippe Champagne, who compared the news to the closing of the Strait of Hormuz blockade, revealed that Mythos was discussed extensively at last week’s International Monetary Fund meeting in Washington. “The difference is that the Strait of Hormuz — we know where it is and we know how large it is,” Champagne told the BBC. “The issue that we’re facing with Anthropic is that it’s the unknown, unknown.” Even the Trump administration is unnerved, reports the Economist.
A week or so after Mythos hit the headlines, Dario Amodei sat down for lunch at Catogna, an Italian restaurant in San Francisco’s Jackson Square, with Financial Times innovation editor John Thornhill. Over two courses accompanied by sparkling water, Thornhill tried valiantly to get a grip on a figure accused by one critic of purveying “disaster-porn-as-marketing-tool” and by a “veteran Silicon Valley investor” as “an extraordinary man, the real genuine article.”
Amodei professed to having been inspired to work in AI by futurist Ray Kurzweil’s influential 2005 book The Singularity Is Near, though he acknowledges it contains some “crazy,” “sci-fi” things. (Physicist Paul Davies described the book as a “breathless romp across the outer reaches of technological possibility” when it was published.) He was especially struck by Kurzweil’s observation that exponential increases in computing power would eventually result in human-level AI.
That vision still grips him. “There’s no end to the rainbow,” Amodei tells Thornhill. “We don’t see anything slowing down. I’m the first to say that it’s going to completely transform the world and we’re underestimating its significance.”
How you react to Kurzweil’s subtitle, “When Humans Transcend Biology,” no doubt mirrors your attitude to this period of seemingly unbridled technological experimentation. Amodei is overwhelmingly optimistic, telling Thornhill that he and his sister, fellow Anthropic founder Daniela Amodei, had long dreamt of doing good together, and “admits to being surprised” that Anthropic has given him what he sees as the chance to do that. Despite his warnings about the need for regulation, he clearly regards the structure and values of Anthropic as a reflection of that early pledge.
“We have an obligation to give back selflessly,” Amodei tells Thornhill at the end of their lunch. “And society does not have to venerate us for doing it.” For Thornhill, those words “make clear that Amodei wants to position himself as one of the good guys in the AI debate. But Amodei’s tone grates with many Silicon Valley critics, who note how his principles align with Anthropic’s commercial interests.”
Perhaps, given that Anthropic on its own can’t (and certainly won’t) significantly slow the rate of AI progress, Amodei’s combination of boosterism and scruples is the best we can expect from him. And the question of whether or not he is the good guy might not matter much if, as he suspects, “open-source models and Chinese developers will be able to replicate Mythos’s capabilities within six to twelve months” — or sooner, given reports this week of unauthorised users having already accessed the model.
Australia’s former e-safety commissioner Alastair MacGibbon was shocked by what he heard at a meeting with Anthropic representatives in Sydney earlier this month. “They are smart people and alarmed by what they have built,” he says. “They are right about the problem, but wrong about the solution.” Leaving Anthropic to choose who it shares information with will leave systems everywhere fatally exposed.
MacGibbon says federal home affairs and cybersecurity minister Tony Burke must act now to “convene critical infrastructure operators, private sector defenders and AI developers.” We must insist local organisations get early access to the Project Glasswing tools, he adds. “Australia cannot defend itself with yesterday’s capability while the frontier moves on without us.”
Australia’s official AI plan, released last November, looks entirely out of step with the challenge. By declaring Australia’s existing legal frameworks — updated “case by case” — are strong enough, it signalled the government had abandoned its earlier plan for mandatory guardrails for high-risk AI systems. Other countries have made a different assessment, wrote lawyers Jake Goldenfein, Christine Parker and Kimberlee Weatherall in the Conversation, citing rules introduced in the European Union, Canada, South Korea, Japan, Brazil and China.
Since then, the Australian Financial Review reported last month, the government has “put tech giants and data centre operators on notice about complying with Australian values and interests when deploying artificial intelligence or face a decade of reactionary regulation.” A decade of after-the-fact regulation doesn’t seem quite what this moment demands. The broad-brush memorandum of understanding that came out of Amodei’s meeting with Anthony Albanese late last month aren’t much more reassuring.
As two New York Times reporters put it, “Major AI breakthroughs are beginning to function less like product launches and more like weapons tests.” Will the threat dramatised by Mythos Preview prompt a rethink in Canberra? •
