Has AI’s ‘ring of power’ now appeared?
How much more evidence do the AI sceptics need?
Most security tooling has historically benefitted defenders more than attackers… We believe the same will hold true here too—eventually… But the transitional period may be tumultuous regardless. By releasing this model initially to a limited group of critical industry partners and open source developers… we aim to enable defenders to begin securing the most important systems before models with similar capabilities become broadly available.
red.anthropic.com, 7/4/26
AI will be the most powerful tool for expanding human capability and potential that anyone has ever seen… It will not all go well. The fear and anxiety about AI is justified; we are in the process of witnessing the largest change to society in a long time, and perhaps ever… My personal takeaway from the last several years, and take on why there has been so much Shakespearean drama between the companies in our field, comes down to this: “Once you see AGI you can’t unsee it.” It has a real “ring of power” dynamic to it, and makes people do crazy things… A lot of the criticism of our industry comes from sincere concern about the incredibly high stakes of this technology.
Sam Altman, 11/4/26
There are now two types of people in the world. Those who’ve seen – or, more precisely, foreseen – what powerful AI can do, and those that haven’t.
A growing number of people – including me – fall into the former category, but we’re currently still in the minority.
So, let me once again attempt to explain the threat godlike AI poses to humanity.
Anecdata
Over the last decade, I’ve mainly earned a living as tech content creator and have written frequently about cybersecurity.
The ins and outs of cybersecurity are, for any normal human, incredibly dull.
But the implications of cyber breaches – for individuals, businesses and nations – can be incredibly dramatic. Global cybercrime cost in excess of US$10 trillion in 2025. That translates to a lot of individuals and businesses who’ve paid a high price for inadequate cyber defences and/or insufficient caution.
Unfortunately, we all now have a lot more to worry about than having our life savings stolen by a Russian hacker.
One of the first thoughts I had after ChatGPT dropped in late 2022 was what the consequences would be if a nation – presumably either China or the US – developed AI powerful enough to overwhelm any cyber defences it encountered.
Modern economies, societies, militaries and critical infrastructure now rely so heavily on interconnected digital systems that an AI capable of breaching cyber defences at scale poses a profound national-security threat.
The havoc anyone with access to such an AI could wreak would be limited only by their imagination.
They could disrupt or disable power grids, ports, rail networks, fuel pipelines, telecoms, satellites, banking systems and emergency services.
They could corrupt logistics software so armies run short of ammunition, food or spare parts.
They could sabotage weapons maintenance systems, falsify sensor data, interfere with air defence networks and degrade command-and-control.
They could weaponise information by hijacking media channels, flooding networks with convincing disinformation and impersonating leaders, triggering panic buying and crashing markets.
They could make people doubt what is real, unleashing deepfake-fuelled chaos and mass public confusion. If such a scenario sounds vaguely familiar, it’s because it’s the plot of Mountainhead.
When that film was released last year, it was marketed and received as a satire. Last May, a review in The Guardian described it as “a super-satirical chamber piece about the deranged, cynical and facetious mindset of the uber-wealthy, the kind of people who think about ancient Rome every day, though not about Nero and his violin.”
Satire typically involves exaggerating something to the point of absurdity.
The prospect of a Mountainhead scenario playing out is no longer absurd. Which brings me to Anthropic’s new cyber-defence-penetrating AI model Claude Mythos Preview, aka Mythos.
Shit just got real
Even if you’re not much interested in AI, you’ve probably already heard about Mythos, so I’ll attempt to keep this brief.
Anthropic claims Mythos is much stronger than other models because it can do more than just spot coding mistakes. Mythos can find serious hidden software flaws, turn some of them into working exploits, and handle longer, multi-step cyber tasks with far less human help.
Given my day job, I know better than most that it’s not unheard of for tech companies to hype their products, but I doubt that’s what’s going on here.
Tech companies hype their products to shift more units. Anthropic has stressed Mythos will not be made generally available. Instead, it is giving early access to a limited group of heavy hitters – AWS, Apple, Cisco, Google, JPMorganChase, Linux Foundation, Microsoft, NVIDIA and Palo Alto Networks – so they can use it to find and fix serious software flaws before malicious actors do.
Many clever, well-informed people are taking the potential threat posed by Mythos extremely seriously.
In the US, Treasury Secretary Scott Bessent and Fed chair Jerome Powell convened an urgent meeting with major bank CEOs to warn them about the model’s cyber-risk potential.
In the UK, the Bank of England, FCA, Treasury and NCSC have been in urgent talks, with major banks, insurers and exchanges expected to be briefed.
The European Central Bank (ECB) is currently gathering information and plans to question eurozone banks about their preparedness.
The Pentagon barred use of Anthropic tools last month because the company refused to loosen safety restrictions on how the military could use its models, particularly in relation to autonomous weapons and domestic surveillance.
But earlier this week, Anthropic co-founder Jack Clark said, “We have a narrow contracting dispute, but I don’t want that to get in the way of the fact that we care deeply about national security. Our position is the government has to know about this stuff ... So absolutely, we’re talking to them about Mythos.”
Militaries, defence contractors, financial institutions, telecoms, utilities, emergency services, energy companies, transport businesses and healthcare systems have spent decades investing billions in cybersecurity, for obvious reasons. Many high IQ individuals work in the field and are handsomely remunerated for doing so.
But it seems the elaborate cyber defences created by mere humans are no longer any match for the latest AI models.
The difficulty of comprehending an exponential take-off
If I told you I launched a business that made a $50,000 loss in its first year, broke even in its second year and generated a $100,000 profit in its third year, you wouldn’t bat an eyelid.
However, if I claimed I launched a business that made a $50,000 loss in its first year, broke even in its second year and generated a $10 billion profit in its third year, you’d likely find that hard to believe.
But dramatic tipping points are not unusual, especially when it comes to technological progress.
Max Roser, “A long-term timeline of technology,” Our World in Data
The graph above is three years old. Given it predicts “Human-level Artificial Intelligence” arriving sometime between 2050-2075, it’s already hopelessly out of date. That further proves my point about the difficulty people have in comprehending just how fast world-changing technologies can progress.
As the graph’s creator, Max Roser noted back in early 2023:
Technology can change the world in ways that are unimaginable until they happen. Switching on an electric light would have been unimaginable for our medieval ancestors. In their childhood, our grandparents would have struggled to imagine a world connected by smartphones and the Internet.
Technological change was extremely slow in the past – the technologies that our ancestors got used to in their childhood were still central to their lives in their old age… we live in a time of extraordinarily fast technological change. For recent generations, it was common for technologies that were unimaginable in their youth to become common later in life.
What is to be done?
We are no longer debating whether advanced AI could become an unfathomably powerful cyber, military and political weapon. The discussion has moved on to how close that weapon now is, who will control it, and whether defensive institutions can adapt before offensive capability outruns them.
Let me attempt to end on a vaguely hopeful note.
It’s difficult but not impossible to imagine competing companies and competing nations agreeing to hasten slowly while they are playing with fire. Fire that increasingly has the potential to, at the very least, massively destabilise societies.
There’s still time to act. But admitting we’ve got a problem is the first step, and not nearly enough people have yet taken it.