Academic

Detecting Multi-Agent Collusion Through Multi-Agent Interpretability

arXiv:2604.01151v1 Announce Type: new Abstract: As LLM agents are increasingly deployed in multi-agent systems, they introduce risks of covert coordination that may evade standard forms of human oversight. While linear probes on model activations have shown promise for detecting deception in single-agent settings, collusion is inherently a multi-agent phenomenon, and the use of internal representations for detecting collusion between agents remains unexplored. We introduce NARCBench, a benchmark for evaluating collusion detection under environment distribution shift, and propose five probing techniques that aggregate per-agent deception scores to classify scenarios at the group level. Our probes achieve 1.00 AUROC in-distribution and 0.60--0.86 AUROC when transferred zero-shot to structurally different multi-agent scenarios and a steganographic blackjack card-counting task. We find that no single probing technique dominates across all collusion types, suggesting that different forms o

A
Aaron Rose, Carissa Cullen, Brandon Gary Kaplowitz, Christian Schroeder de Witt · · 1 min read · 7 views
#cs.AI #cs.LG #cs.MA

arXiv:2604.01151v1 Announce Type: new Abstract: As LLM agents are increasingly deployed in multi-agent systems, they introduce risks of covert coordination that may evade standard forms of human oversight. While linear probes on model activations have shown promise for detecting deception in single-agent settings, collusion is inherently a multi-agent phenomenon, and the use of internal representations for detecting collusion between agents remains unexplored. We introduce NARCBench, a benchmark for evaluating collusion detection under environment distribution shift, and propose five probing techniques that aggregate per-agent deception scores to classify scenarios at the group level. Our probes achieve 1.00 AUROC in-distribution and 0.60--0.86 AUROC when transferred zero-shot to structurally different multi-agent scenarios and a steganographic blackjack card-counting task. We find that no single probing technique dominates across all collusion types, suggesting that different forms of collusion manifest differently in activation space. We also find preliminary evidence that this signal is localised at the token level, with the colluding agent's activations spiking specifically when processing the encoded parts of their partner's message. This work takes a step toward multi-agent interpretability: extending white-box inspection from single models to multi-agent contexts, where detection requires aggregating signals across agents. These results suggest that model internals provide a complementary signal to text-level monitoring for detecting multi-agent collusion, particularly for organisations with access to model activations. Code and data are available at https://github.com/aaronrose227/narcbench.

Executive Summary

This article introduces NARCBench, a benchmark for evaluating collusion detection in multi-agent systems. The authors propose five probing techniques that aggregate per-agent deception scores to classify scenarios at the group level, achieving high AUROC scores in various multi-agent scenarios. The study suggests that model internals provide a complementary signal to text-level monitoring for detecting multi-agent collusion, particularly for organisations with access to model activations. However, the findings also indicate that no single probing technique dominates across all collusion types, suggesting that collusion may manifest differently in activation space.

Key Points

  • ▸ NARCBench is introduced as a benchmark for evaluating collusion detection in multi-agent systems.
  • ▸ Five probing techniques are proposed to aggregate per-agent deception scores for group-level classification.
  • ▸ The study achieves high AUROC scores in various multi-agent scenarios, including a steganographic blackjack card-counting task.

Merits

Contribution to Multi-Agent Interpretability

The study extends white-box inspection from single models to multi-agent contexts, providing a novel approach to detecting collusion in complex systems.

Development of NARCBench Benchmark

The introduction of NARCBench provides a standardized framework for evaluating collusion detection in multi-agent systems, enabling comparisons across different techniques and scenarios.

Demerits

Limited Generalizability

The study's findings may not generalize to all multi-agent scenarios, particularly those with significant structural differences from the ones explored in the research.

Need for Further Investigation

The study suggests that collusion may manifest differently in activation space, highlighting the need for further investigation into the underlying mechanisms and potential countermeasures.

Expert Commentary

The study's findings have significant implications for the development of multi-agent systems, particularly in scenarios where covert coordination could lead to adverse consequences. While the proposed probing techniques demonstrate promising results, further investigation is needed to fully understand the underlying mechanisms of collusion and to develop more effective countermeasures. Additionally, the study highlights the importance of model interpretability in multi-agent contexts, which is a critical area of research that requires further attention.

Recommendations

  • ✓ Future research should focus on developing more robust and generalizable probing techniques that can detect collusion in a wider range of multi-agent scenarios.
  • ✓ Organisations should consider implementing collusion detection systems to prevent covert coordination in multi-agent scenarios, particularly in high-stakes applications.

Sources

Original: arXiv - cs.AI

Related Articles