Political and scientific debate over the risks of “superintelligent artificial intelligence” (ASI) intensified after a session held in the British Parliament, during which officials and researchers warned that these systems could reach a level of capability that would be difficult for humans to control. Des Browne, the former British defense secretary, went so far as to say that the potential threat could equal the danger of nuclear war, and possibly exceed it.
The session was attended by the Control AI coalition, which calls for international rules governing artificial intelligence and supports a bill being debated by the British Parliament to ban the development of superintelligent artificial intelligence. Stuart Russell, a computer scientist at the University of California, Berkeley, warned of the possibility of a large-scale catastrophe and of a scenario in which humanity loses control of artificial intelligence systems irreversibly.
Varying Estimates of the Scale of the Risk
Evan Hubinger, who is responsible for alignment research at Anthropic, sparked controversy after saying that there is a probability of more than 10% that this technology could kill all humans during the next decade. This figure does not represent a consensus among experts, but rather an estimate attributed to Hubinger within a broader discussion about future risks.
This coincided with the resignation of 28-year-old Jacob Coxon from Anthropic, after he had previously worked at OpenAI. Coxon said that the two companies were not acting responsibly and were caught in a race to reach superintelligent artificial intelligence first, but he expressed hope that the major American laboratories could coordinate to slow this race. Experts do not agree on when these capabilities will emerge; some speak of a few years, while others expect more than a decade.
From Warnings to International Rules
Labour MP Darren Jones called for “a multilateral treaty for the orderly and safe development of superintelligence,” stressing that the objective, according to his proposal, is not to halt research and innovation but to prioritize safety as development accelerates. The British government also expressed concern that Anthropic had not submitted its latest model, Mythos 5.1, to the UK AI Security Institute for pre-release testing. The Cabinet Office said that a limited number of American institutions had been given access to the model, while cooperation between the institute and companies continued.
In the United States, Senator Bernie Sanders called on Congress to act, citing a poll that said 81% of Americans believe politicians need to take action on artificial intelligence. Sanders links the debate to the future of the economy, the environment, democracy, and privacy.
Why Does This News Matter?
The debate reveals a gap between long-term risks whose probabilities are difficult to verify and existing risks that can be observed today. Andrew Rogoyski warned that current systems remain far from human flexibility and that advanced artificial intelligence technologies may be costly or of limited benefit. David Barber said that the practical problem could worsen when systems gain unrestricted access to the internet and different systems, while Sandra Wachter pointed to disinformation, environmental impact, and jobs as real and urgent issues.
Gary Marcus believes that the decisive difference will be between superintelligent artificial intelligence aligned with human interests and another form that is not aligned with them, while acknowledging that the current debate includes media noise and a lack of caution and responsibility. Thus, the source does not establish that a human-extinction scenario is imminent or certain, but it shows that the disagreement over the level of risk is no longer merely academic; it has become linked to legislation, pre-release testing, and limits on models’ access to systems.