CA-HFP: Curvature-Aware Heterogeneous Federated Pruning with Model Reconstruction
arXiv:2603.12591v1 Announce Type: new Abstract: Federated learning on heterogeneous edge devices requires personalized compression while preserving aggregation compatibility and stable convergence. We present Curvature-Aware Heterogeneous Federated Pruning (CA-HFP), a practical framework that enables each client perform structured, device-specific pruning guided by a curvature-informed significance score, and subsequently maps its compact submodel back into a common global parameter space via a lightweight reconstruction. We derive a convergence bound for federated optimization with multiple local SGD steps that explicitly accounts for local computation, data heterogeneity, and pruning-induced perturbations; from which a principled loss-based pruning criterion is derived. Extensive experiments on FMNIST, CIFAR-10, and CIFAR-100 using VGG and ResNet architectures under varying degrees of data heterogeneity demonstrate that CA-HFP preserves model accuracy while significantly reducing pe
arXiv:2603.12591v1 Announce Type: new Abstract: Federated learning on heterogeneous edge devices requires personalized compression while preserving aggregation compatibility and stable convergence. We present Curvature-Aware Heterogeneous Federated Pruning (CA-HFP), a practical framework that enables each client perform structured, device-specific pruning guided by a curvature-informed significance score, and subsequently maps its compact submodel back into a common global parameter space via a lightweight reconstruction. We derive a convergence bound for federated optimization with multiple local SGD steps that explicitly accounts for local computation, data heterogeneity, and pruning-induced perturbations; from which a principled loss-based pruning criterion is derived. Extensive experiments on FMNIST, CIFAR-10, and CIFAR-100 using VGG and ResNet architectures under varying degrees of data heterogeneity demonstrate that CA-HFP preserves model accuracy while significantly reducing per-client computation and communication costs, outperforming standard federated training and existing pruning-based baselines.
Executive Summary
This study introduces Curvature-Aware Heterogeneous Federated Pruning (CA-HFP), a novel framework for personalized compression in federated learning. CA-HFP leverages curvature-informed significance scores to guide device-specific pruning, enabling the creation of compact submodels that can be accurately reconstructed in a common global parameter space. Through extensive experiments on various datasets and architectures, the authors demonstrate CA-HFP's ability to preserve model accuracy while significantly reducing computation and communication costs. The framework's performance is compared to standard federated training and existing pruning-based approaches, with CA-HFP consistently outperforming these baselines. This research has significant implications for the efficient deployment of federated learning models on resource-constrained edge devices, where data heterogeneity and computation constraints are pressing concerns.
Key Points
- ▸ CA-HFP introduces curvature-aware pruning for personalized compression in federated learning.
- ▸ The framework uses device-specific pruning and lightweight reconstruction to preserve model accuracy.
- ▸ CA-HFP significantly reduces computation and communication costs compared to standard federated training and pruning-based approaches.
Merits
Strength
The study provides a comprehensive theoretical framework for curvature-aware pruning, including a principled loss-based pruning criterion and a convergence bound for federated optimization.
Strength
The authors demonstrate CA-HFP's efficacy through extensive experiments on various datasets and architectures, showcasing its ability to preserve model accuracy and reduce computation and communication costs.
Strength
The framework's performance is compared to multiple baselines, providing a clear evaluation of CA-HFP's advantages over existing approaches.
Demerits
Limitation
The study focuses on reducing computation and communication costs, but does not explore the potential impact of CA-HFP on model interpretability or explainability.
Limitation
The authors assume that the global parameter space is known, which may not be feasible in practice, particularly in scenarios with large numbers of clients or complex model architectures.
Expert Commentary
The study's strengths lie in its comprehensive theoretical framework and extensive experimental evaluation. However, the authors could have explored the potential impact of CA-HFP on model interpretability and explainability, which is a growing concern in the field of deep learning. Additionally, the assumption that the global parameter space is known may not be feasible in practice. Nevertheless, CA-HFP presents a promising approach to efficient federated learning on heterogeneous edge devices, and its performance advantages over existing baselines make it an attractive solution for real-world applications.
Recommendations
- ✓ Future research should investigate the application of CA-HFP to more complex model architectures and larger-scale federated learning scenarios.
- ✓ The authors should explore methods to overcome the assumption that the global parameter space is known, potentially through the development of hybrid frameworks that combine CA-HFP with other federated learning approaches.