Certified Per-Instance Unlearning Using Individual Sensitivity Bounds
arXiv:2602.15602v1 Announce Type: new Abstract: Certified machine unlearning can be achieved via noise injection leading to differential privacy guarantees, where noise is calibrated to worst-case sensitivity. Such conservative calibration often results in performance degradation, limiting practical applicability. In this work, we investigate an alternative approach based on adaptive per-instance noise calibration tailored to the individual contribution of each data point to the learned solution. This raises the following challenge: how can one establish formal unlearning guarantees when the mechanism depends on the specific point to be removed? To define individual data point sensitivities in noisy gradient dynamics, we consider the use of per-instance differential privacy. For ridge regression trained via Langevin dynamics, we derive high-probability per-instance sensitivity bounds, yielding certified unlearning with substantially less noise injection. We corroborate our theoretical
arXiv:2602.15602v1 Announce Type: new Abstract: Certified machine unlearning can be achieved via noise injection leading to differential privacy guarantees, where noise is calibrated to worst-case sensitivity. Such conservative calibration often results in performance degradation, limiting practical applicability. In this work, we investigate an alternative approach based on adaptive per-instance noise calibration tailored to the individual contribution of each data point to the learned solution. This raises the following challenge: how can one establish formal unlearning guarantees when the mechanism depends on the specific point to be removed? To define individual data point sensitivities in noisy gradient dynamics, we consider the use of per-instance differential privacy. For ridge regression trained via Langevin dynamics, we derive high-probability per-instance sensitivity bounds, yielding certified unlearning with substantially less noise injection. We corroborate our theoretical findings through experiments in linear settings and provide further empirical evidence on the relevance of the approach in deep learning settings.
Executive Summary
This article presents an innovative approach to certified machine unlearning by introducing per-instance sensitivity bounds, which enables adaptive noise calibration tailored to individual data points. The authors derive high-probability sensitivity bounds for ridge regression trained via Langevin dynamics, resulting in certified unlearning with less noise injection. The work demonstrates the efficacy of per-instance differential privacy in addressing the challenge of noise calibration. Although the article makes significant contributions to the field of machine learning, its scope is limited to specific regression settings, and further research is necessary to explore its applicability to broader machine learning models. The findings have significant implications for the development of trustworthy machine learning systems that can provide guarantees on data removal.
Key Points
- ▸ The article proposes per-instance sensitivity bounds for certified machine unlearning.
- ▸ The authors derive high-probability sensitivity bounds for ridge regression trained via Langevin dynamics.
- ▸ The approach enables adaptive noise calibration tailored to individual data points.
Merits
Strength in theoretical foundations
The article establishes a solid theoretical foundation for per-instance sensitivity bounds, providing a clear understanding of the underlying mechanisms and guarantees.
Innovative approach to noise calibration
The authors' adaptive noise calibration approach has the potential to improve the practical applicability of certified machine unlearning.
Demerits
Limited scope
The article is focused on specific regression settings, and further research is necessary to explore its applicability to broader machine learning models.
Experimental limitations
The experimental evaluation is limited to linear settings, and more comprehensive experiments are necessary to validate the approach in deep learning settings.
Expert Commentary
The article presents an innovative approach to certified machine unlearning, which has the potential to address the challenge of noise calibration. However, the scope of the work is limited to specific regression settings, and further research is necessary to explore its applicability to broader machine learning models. The findings have significant implications for the development of trustworthy machine learning systems. To further improve the approach, it is essential to conduct more comprehensive experiments in deep learning settings and explore its applicability to more complex machine learning models. Additionally, the authors should consider extending the approach to other machine learning algorithms and settings.
Recommendations
- ✓ Conduct more comprehensive experiments in deep learning settings to validate the approach.
- ✓ Explore the applicability of the approach to more complex machine learning models and algorithms.
