COMBA: Cross Batch Aggregation for Learning Large Graphs with Context Gating State Space Models
arXiv:2602.17893v1 Announce Type: new Abstract: State space models (SSMs) have recently emerged for modeling long-range dependency in sequence data, with much simplified computational costs than modern alternatives, such as transformers. Advancing SMMs to graph structured data, especially for large graphs, is a significant challenge because SSMs are sequence models and the shear graph volumes make it very expensive to convert graphs as sequences for effective learning. In this paper, we propose COMBA to tackle large graph learning using state space models, with two key innovations: graph context gating and cross batch aggregation. Graph context refers to different hops of neighborhood for each node, and graph context gating allows COMBA to use such context to learn best control of neighbor aggregation. For each graph context, COMBA samples nodes as batches, and train a graph neural network (GNN), with information being aggregated cross batches, allowing COMBA to scale to large graphs.
arXiv:2602.17893v1 Announce Type: new Abstract: State space models (SSMs) have recently emerged for modeling long-range dependency in sequence data, with much simplified computational costs than modern alternatives, such as transformers. Advancing SMMs to graph structured data, especially for large graphs, is a significant challenge because SSMs are sequence models and the shear graph volumes make it very expensive to convert graphs as sequences for effective learning. In this paper, we propose COMBA to tackle large graph learning using state space models, with two key innovations: graph context gating and cross batch aggregation. Graph context refers to different hops of neighborhood for each node, and graph context gating allows COMBA to use such context to learn best control of neighbor aggregation. For each graph context, COMBA samples nodes as batches, and train a graph neural network (GNN), with information being aggregated cross batches, allowing COMBA to scale to large graphs. Our theoretical study asserts that cross-batch aggregation guarantees lower error than training GNN without aggregation. Experiments on benchmark networks demonstrate significant performance gains compared to baseline approaches. Code and benchmark datasets will be released for public access.
Executive Summary
The article proposes COMBA, a novel approach for learning large graphs using state space models. COMBA introduces graph context gating and cross-batch aggregation to tackle the challenges of large graph learning. The method samples nodes as batches, trains a graph neural network, and aggregates information across batches, enabling scalability to large graphs. Theoretical studies and experiments demonstrate the effectiveness of COMBA, showing significant performance gains compared to baseline approaches.
Key Points
- ▸ Introduction of graph context gating to learn control of neighbor aggregation
- ▸ Cross-batch aggregation to scale to large graphs
- ▸ Theoretical guarantee of lower error compared to training GNN without aggregation
Merits
Scalability
COMBA's cross-batch aggregation allows it to scale to large graphs, making it a promising approach for real-world applications
Theoretical Guarantee
The theoretical study provides a guarantee of lower error, adding confidence to the proposed method
Demerits
Complexity
The introduction of graph context gating and cross-batch aggregation may add complexity to the model, potentially increasing computational costs
Expert Commentary
The proposed COMBA approach demonstrates a significant advancement in the field of graph learning. By introducing graph context gating and cross-batch aggregation, the authors have addressed a major challenge in scaling state space models to large graphs. The theoretical guarantee of lower error and the experimental results demonstrating significant performance gains make COMBA a promising approach for real-world applications. However, further research is needed to fully explore the potential of COMBA and to address potential limitations, such as increased computational costs.
Recommendations
- ✓ Further experimentation on various graph datasets to fully evaluate the performance of COMBA
- ✓ Investigation into the application of COMBA to other graph learning tasks, such as graph generation and graph clustering
