Differential Privacy in Two-Layer Networks: How DP-SGD Harms Fairness and Robustness
arXiv:2603.04881v1 Announce Type: new Abstract: Differentially private learning is essential for training models on sensitive data, but empirical studies consistently show that it can degrade performance, introduce fairness issues like disparate impact, and reduce adversarial robustness. The theoretical underpinnings of these phenomena in modern, non-convex neural networks remain largely unexplored. This paper introduces a unified feature-centric framework to analyze the feature learning dynamics of differentially private stochastic gradient descent (DP-SGD) in two-layer ReLU convolutional neural networks. Our analysis establishes test loss bounds governed by a crucial metric: the feature-to-noise ratio (FNR). We demonstrate that the noise required for privacy leads to suboptimal feature learning, and specifically show that: 1) imbalanced FNRs across classes and subpopulations cause disparate impact; 2) even in the same class, noise has a greater negative impact on semantically long-t
arXiv:2603.04881v1 Announce Type: new Abstract: Differentially private learning is essential for training models on sensitive data, but empirical studies consistently show that it can degrade performance, introduce fairness issues like disparate impact, and reduce adversarial robustness. The theoretical underpinnings of these phenomena in modern, non-convex neural networks remain largely unexplored. This paper introduces a unified feature-centric framework to analyze the feature learning dynamics of differentially private stochastic gradient descent (DP-SGD) in two-layer ReLU convolutional neural networks. Our analysis establishes test loss bounds governed by a crucial metric: the feature-to-noise ratio (FNR). We demonstrate that the noise required for privacy leads to suboptimal feature learning, and specifically show that: 1) imbalanced FNRs across classes and subpopulations cause disparate impact; 2) even in the same class, noise has a greater negative impact on semantically long-tailed data; and 3) noise injection exacerbates vulnerability to adversarial attacks. Furthermore, our analysis reveals that the popular paradigm of public pre-training and private fine-tuning does not guarantee improvement, particularly under significant feature distribution shifts between datasets. Experiments on synthetic and real-world data corroborate our theoretical findings.
Executive Summary
This article examines the impact of differential privacy on fairness and robustness in two-layer neural networks. The authors introduce a unified feature-centric framework to analyze the effects of differentially private stochastic gradient descent (DP-SGD) on feature learning dynamics. They demonstrate that the noise required for privacy leads to suboptimal feature learning, causing disparate impact and increased vulnerability to adversarial attacks. The study highlights the limitations of public pre-training and private fine-tuning, particularly under significant feature distribution shifts between datasets.
Key Points
- ▸ Differential privacy can degrade performance and introduce fairness issues
- ▸ Noise required for privacy leads to suboptimal feature learning
- ▸ Public pre-training and private fine-tuning may not guarantee improvement
Merits
Novel Framework
The authors introduce a unified feature-centric framework to analyze the effects of DP-SGD on feature learning dynamics
Demerits
Limited Scope
The study focuses on two-layer ReLU convolutional neural networks, which may not be representative of all neural network architectures
Expert Commentary
This study provides valuable insights into the limitations of differential privacy in deep learning. The authors' feature-centric framework offers a nuanced understanding of how noise affects feature learning dynamics, leading to disparate impact and reduced robustness. However, the study's findings should be considered in the context of the broader machine learning landscape, where the trade-offs between privacy, fairness, and robustness are complex and multifaceted. Further research is needed to develop more effective differentially private learning algorithms that balance these competing priorities.
Recommendations
- ✓ Developing more robust differentially private learning algorithms
- ✓ Conducting further research on the trade-offs between privacy, fairness, and robustness in machine learning