What’s Covered?
This technical report offers both a risk taxonomy and an agenda for future research and mitigation. The authors classify multi-agent failures into three distinct types based on agent incentives and interaction failures:
- Miscoordination (e.g., conflicting plans, bottlenecks, unintended interference)
- Conflict (e.g., resource competition, escalation, winner-takes-all behavior)
- Collusion (e.g., price fixing, market manipulation, cartel-like behavior)
In addition to failure modes, the report identifies seven cross-cutting risk factors that make these failures more likely or more severe:
- Information asymmetries – gaps or distortions in what agents know
- Network effects – lock-in, tipping points, or runaway dominance
- Selection pressures – incentives that evolve agent behavior
- Destabilising dynamics – feedback loops or race conditions
- Commitment problems – inability to bind future behavior
- Emergent agency – group-level behavior that isn’t predictable from the parts
- Multi-agent security – adversarial tactics like infiltration or deception
For each failure type and risk factor, the authors provide illustrative examples, theoretical background, and preliminary mitigation strategies. These are drawn from game theory, economic regulation, cybersecurity, and empirical studies of existing agent behavior. The report also includes several case studies on coordination failures in driving, the spread of misinformation through agent networks, and adversarial manipulation of overseer systems.
đź’ˇ Why it matters?
This resource reframes AI safety to reflect a more networked future—where risks don’t just come from rogue individual systems, but from interaction failures among many. As AI agents are increasingly tasked with interacting, negotiating, or competing on behalf of humans, overlooking multi-agent dynamics would be like regulating individual cars without considering traffic. The report makes clear that many single-agent safeguards—alignment, interpretability, containment—won’t translate neatly to these emergent settings. What’s needed now is a shift in safety mindset, tools, and governance mechanisms.
What’s Missing?
While rich in concepts, the report offers limited practical tooling or metrics for assessing multi-agent risk in deployed systems. Readers looking for concrete guidelines on how to integrate these ideas into AI development pipelines may find the document more foundational than operational. There’s also little discussion of legal implications—like antitrust enforcement in automated marketplaces—or examples of regulators successfully grappling with agent interactions (as in algorithmic trading). Finally, although governance and ethics are acknowledged, the report largely defers to future work in those areas.
Best For:
This report is a must-read for AI safety researchers, multi-agent systems specialists, and policy advisors focused on emerging AI governance challenges. It’s especially useful for those building or overseeing systems involving autonomous agents interacting at scale—across economics, defense, infrastructure, or simulation environments. Also valuable for academics or think tanks crafting risk taxonomies or looking to plug conceptual gaps in standard safety approaches.
Source Details:
Hammond et al. (2025). Multi-Agent Risks from Advanced AI. Cooperative AI Foundation, Technical Report #1.
With contributions from 50+ researchers across academia, industry, and independent labs—including DeepMind, Anthropic, Carnegie Mellon, and Harvard—this document represents a major collaborative effort to anchor multi-agent AI risk as a core concern in the safety and governance discourse. The report was released as an open-access arXiv preprint (arXiv:2502.14143v1), signaling its role as both a research milestone and a policy-shaping intervention. The Cooperative AI Foundation coordinated the project to align with its broader mission of advancing cooperative capabilities in AI.
