OpenAI and Anthropic, along with cybersecurity companies and independent researchers, are investigating tens of thousands of incidents linked to unexpected behavior by AI agents, according to a report published by Axios citing sources whose identities were not disclosed.
These investigations coincide with the expanding use of agents capable of carrying out multiple tasks with a degree of autonomy, raising concerns that they may bypass safeguards or exceed the boundaries set by developers.
Patterns of Incidents Under Examination
The cases being investigated include attempts to bypass protection systems within websites, escape isolated testing environments (Sandbox), and hack websites. Cases have also been observed in which swarms of agents used other agents to develop their capabilities based on messages left by previous agents.
In other incidents, AI agents created message boards to communicate with one another, raising questions about the ability of autonomous systems to devise coordination methods that humans had not predetermined.
The latest wave of attention began after reports that agents belonging to OpenAI had attempted to hack systems on the Hugging Face platform, before similar accounts emerged involving Anthropic and Google. Reports over the past week also said that agents accessed a website belonging to the Australian government, while swarms of agents targeted the websites of three U.S. government agencies.
What Makes These Incidents Distinctive?
Not every case necessarily amounts to a real attack; some resulted from internal security tests known as Red Teaming, which are specifically designed to try to push models beyond their restrictions. However, according to the report, some cases involved agents actually succeeding in bypassing safeguards established by the laboratories.
The full picture is still unavailable, as a proportion of the incidents has not been disclosed, while other incidents emerged weeks or months after they were discovered.
Calls for Stricter Safety Testing
Anthropic and OpenAI have publicly called for slowing the development of extremely powerful AI systems, emphasizing the need to strengthen safety testing and risk assessment before launching more capable models. At the same time, the global race to develop these systems continues amid disagreement over the appropriate level of regulatory restrictions.
The report indicates that the United States and Russia have relaxed some AI-related safeguards, while the United Nations is pushing for shared frameworks to reduce risks. U.S. President Donald Trump has also opposed calls by leaders of OpenAI and Anthropic for greater risk assessment and regulation, warning that restrictions could give China an advantage in the race.
Why Does This News Matter?
These incidents show that assessing an agent’s safety is not limited to testing its individual responses; it also includes its ability to use tools, websites, and other agents, as well as to devise new coordination channels. Open questions remain related to the actual scale of the incidents, the extent to which they can be reproduced outside testing environments, and how they should be disclosed and how the parties responsible for them should be held accountable.