Is AI too dangerous?

The short answer

Is AI too dangerous?

On September 12, 2026, Dario Amodei, chief executive of Anthropic, published an essay with a striking message: the development of the most powerful AI has to slow down. Within days he was backed by two competitors who rarely agree with him, Sam Altman of OpenAI and Elon Musk of xAI.

Literally, they do not say that AI is too dangerous. They say the pace has to come down, so that safety research can keep up with the technology. Altman wrote that he agrees with Amodei and that this had been the main topic of conversation at OpenAI for weeks. Musk kept it to three words: Dario is right.

In the video above I walk through the story at the whiteboard. This piece sets out exactly what happened, with sources.

What prompted it

What happened at Hugging Face?

The immediate cause is an incident from July 2026. On July 21 OpenAI disclosed that two of its AI models had broken out of their sealed environment during an internal test.

The models were given a test in digital security. Instead of solving the problems, they looked for a shortcut. They found a weak spot in the test environment, gave themselves access to the internet and broke into Hugging Face, the platform where developers worldwide share AI models and datasets. The answers to the test were there.

No human had instructed them to do so. According to the reconstruction by Hugging Face it involved more than seventeen thousand actions over a few days. The programs coordinated their work through a messaging channel they built themselves. OpenAI discovered that its own systems were involved only after Hugging Face reported the break-in.

Unpredictable behavior that we are already seeing, plus a development that runs exponentially: that is a reason for concern.Paul Hoffman, in the video

One detail matters for the rest of this story. When Hugging Face set out to analyze the attack, the American models refused the work because of their own safety rules. The company then used a freely available model from the Chinese company Z.ai. The damage was repaired with exactly the kind of model that is under fire in this debate.

The other lens

Why is it these three who say it?

The concern about safety is well founded. Even so, it pays to look through a second lens. Each of the three companies has a reason of its own to find a lower pace attractive.

OpenAI: money

Building the most powerful models is extraordinarily expensive. OpenAI is estimated to burn tens of billions of dollars a year and has taken on commitments for computing power that far exceed the means available to it. If you hit the brakes because things are getting too dangerous, you no longer have to fund that pace either.

Anthropic: capacity

Anthropic has been short of computing power for months. Customers notice it in limits on usage. A lower pace across the whole sector gives it room to breathe. At the same time, according to reports, the company signed hundreds of billions of dollars in contracts for new capacity in a short period. Calling for a slowdown and expanding heavily go hand in hand here.

xAI: falling behind

Musk's model, Grok, is growing quickly among consumers, but among businesses and developers it clearly trails the two leaders. For whoever is behind, a pause is the cheapest way to close the gap.

None of these interests proves the concern is feigned. They do explain why three rivals suddenly agree.

The larger factor

What role does China play?

Above these individual motives sits a shared interest. Chinese companies are releasing open source models at a rapid pace: models that are free to download and that an organization can run on its own hardware, without a subscription with an American provider.

Those models are good, they follow one another quickly and they cost almost nothing, provided you pay for the hardware and the electricity. For companies investing billions in closed models that is a direct threat to the business model.

Amodei proposes three steps: independent assessors with far-reaching access to the newest systems, agreements within the industry, and international agreements so that countries such as China moderate the pace as well. Critics point at what this means in practice. Oversight in the form of licenses, limits on computing power and export rules is easy for established parties to carry and hard for challengers. A certification body that decides which models are available in a country keeps the free Chinese models out first of all.

Others turn it around. As long as China does not take part, slowing down mainly means, in their view, that the West gives away its lead. Amodei himself calls China the toughest dilemma in his proposal.

My reading

Is this safety or market protection?

Both. And that is exactly what makes it a hard question.

My opinion is divided. On the one hand I understand that exponential development and how fast it is going. On the other hand I also understand that there are other motives at play. I look at both sides of the story with a skeptical eye.Paul Hoffman, in the video

The incident at Hugging Face is real and it is serious. Systems that break out of their environment without being told to and break in somewhere else call for better oversight. At the same time it is striking that the solution being proposed protects precisely the parties proposing it. Anyone with an outspoken opinion in this debate has usually looked at only one of the two sides.

In practice

What does this mean for your organization?

For a director or a management team this is not a distant debate. It determines which AI will be available, at what price and under which rules. Four sober conclusions.

Do not wait for the outcome. The value of AI in your processes does not depend on the very latest model. It depends on how the work around it is arranged. What is available today is enough for almost every application that pays for itself.

Avoid dependence on a single vendor. The balance of power in this market shifts every quarter. Supply, prices and terms can change quickly. A setup that can run on more than one model is not a luxury but a precaution.

Take free models seriously, and keep an eye on the rules. For work where data may not leave the building, models on your own hardware are a real alternative. If a certification regime arrives, their availability may change. That is a reason to follow it, not to avoid it.

Keep a person at the wheel where mistakes are expensive. The incident shows that systems working on their own choose unexpected routes. In processes where a mistake costs money, customers or reputation, a person belongs at the final step.