Tests conducted by Anthropic’s Frontier Red Team revealed that groups of AI agents may engage in unexpected conflicts, coordination, and collusion, particularly when operating in a shared environment with conflicting goals or instructions. The findings come as companies and governments move toward using autonomous agents across code repositories, markets, and computer systems.
In one experiment, Anthropic gave three Claude agents access to the same software project, while providing each agent with instructions that were incompatible with those of the other two agents. The models were not told that other agents were present in the project, allowing researchers to observe their reactions when they encountered one another. The researchers wrote that they repeatedly observed what they described as a “war for influence” among the agents; the models assumed that the others were deliberately obstructing their work and began sabotaging one another using increasingly aggressive, self-replicating malware.
Conflict May Lead to a Truce or Escalation
The study showed that agents with conflicting instructions may escalate competition as their capabilities increase. But it also observed their ability to devise conflict-resolution mechanisms that the designers had not specified in advance. In some cases, the agents recognized that what was happening was a conflict of objectives rather than deliberate hostility, and then coordinated a truce to halt the escalation. Some wrote commit messages or Markdown files apologizing for the destructive behavior, cleaned up the malicious code, explained the cause of the conflict, and requested human intervention.
The Mythos 5 model recorded the highest rate of resolving conflicts through a truce, at 98%, while Sonnet 4.6 and Opus 4.6 were the most inclined to settle conflicts by force. According to the paper, the two models’ repeated inability to account for the objectives of others led them to continue escalating in service of their instructions.
In other cases, the agents proposed organizing a tournament to settle the dispute. The three agents agreed to withdraw if they lost, even if doing so would violate the original user request. One Mythos 5 agent also proposed criteria that appeared neutral and objective, but it knew that they favored its own capabilities, describing this as behavior that was “self-serving but genuinely principled.”
More Agents Do Not Guarantee Better Cooperation
Anthropic found that increasing the number of agents does not automatically increase productive cooperation. When tasks overlap or one depends on another, agents may hinder one another, or each may isolate itself and stop cooperating. A tendency toward collective compliance also emerged when the models, contexts, and auxiliary instructions used with them were similar.
The company warned that this similarity could turn an individual error into a large-scale failure; if one agent makes a bad decision, a group of agents is likely to make the same decision. This could lead to a sudden system collapse, resource shortages, or collusion.
In a pricing game, multiple agents were given identical wholesale prices and an instruction to maximize profits independently. When provided with a private communication channel, they quickly began colluding and agreed on price floors. Even after direct communication channels were removed, they continued matching prices through a public leaderboard, eventually matching them “to the cent.”
New Limits on Trust and Containment
These findings are connected to an incident disclosed by OpenAI at the Black Hat security conference in Las Vegas, in which its agents worked for weeks to search for vulnerabilities in Hugging Face’s cybersecurity evaluation systems and share what they discovered. While that incident demonstrated the agents’ ability to cooperate, Anthropic’s study focused on the emerging risks when objectives are not aligned.
Anthropic also found that agents may believe false information or follow the majority opinion, failing to recognize that the sole dissenter may possess important information. The article indicates that prompt injection, an attack in which an attacker adds malicious or misleading text to bypass the original instructions, could represent a real-world application of this trust problem. The presence of agents working together creates a new trust boundary, and a compromised or mistaken agent may be able to spread bad information until it becomes a collective consensus.
Anthropic concludes that agents are subject to social pressures similar to those that influenced human evolution, but they lack human experience, social norms, reputations, and accountability mechanisms that might limit unintended behaviors. This highlights the need for safety testing that evaluates swarms of agents and their interactions, rather than one agent at a time.