Privacy-Preserving Machine Learning and the Trade-Off Between Model Utility and Data Protection in Federated and Decentralized Learning Frameworks
Keywords:
Privacy-preserving machine learning, federated learning, decentralized learning, differential privacy, homomorphic encryption, model utility, data protection, secure multi-party computationAbstract
With the growing concerns over data privacy, privacy-preserving machine learning (PPML) has emerged as a significant field, particularly in federated and decentralized learning frameworks. These frameworks aim to maintain data privacy by processing data locally while collaboratively training models across multiple nodes. However, there is an inherent trade-off between data protection and model utility—enhancing privacy often leads to reduced model performance. This paper explores various privacy-preserving techniques in federated and decentralized learning, including differential privacy, homomorphic encryption, and secure multi-party computation. We examine key research before 2023, discussing their contributions, limitations, and trade-offs in balancing privacy and model accuracy. Additionally, we present comparative analyses of different privacy mechanisms and their impact on model performance using experimental data. The paper concludes by identifying future research directions and highlighting the critical challenges in achieving optimal privacy-utility trade-offs.
References
Bonawitz, K., et al. Practical Secure Aggregation for Privacy-Preserving Machine Learning. Proceedings of ACM CCS, 2017.
Kairouz, P., et al. Advances and Open Problems in Federated Learning. Foundations and Trends in Machine Learning, 2021.
McMahan, B., et al. Communication-Efficient Learning of Deep Networks from Decentralized Data. AISTATS, 2017.
Shokri, R., & Shmatikov, V. Privacy-Preserving Deep Learning. ACM CCS, 2015.
Li, T., et al. Federated Learning: Challenges, Methods, and Future Directions. IEEE Signal Processing Magazine, 2020.
Abadi, M., et al. Deep Learning with Differential Privacy. ACM CCS, 2016.
Geyer, R. C., et al. Differentially Private Federated Learning: A Client-Level Perspective. arXiv, 2017.
Papernot, N., et al. Semi-supervised Knowledge Transfer for Deep Learning from Private Training Data. ICLR, 2017.
Melis, L., et al. Exploiting Unintended Feature Leakage in Collaborative Learning. IEEE S&P, 2019.
Phong, L., et al. Privacy-Preserving Deep Learning via Additively Homomorphic Encryption. IEEE Transactions, 2018.
Truex, S., et al. A Hybrid Approach to Privacy-Preserving Federated Learning. arXiv, 2019.
Hynes, N., et al. Efficient Deep Learning on Multi-Party Encrypted Data. NeurIPS, 2018.
Dwork, C. Differential Privacy. Proceedings of ICALP, 2006.
Goldreich, O. Secure Multi-Party Computation. Cambridge University Press, 2004.
Mohassel, P., & Zhang, Y. SecureML: A System for Scalable Privacy-Preserving Machine Learning. IEEE S&P, 2017.
Liu, Y., et al. Decentralized Machine Learning: A Comprehensive Survey. IEEE Access, 2022.
Hardy, S., et al. Private Federated Learning on Vertically Partitioned Data. arXiv, 2018.
Jayaraman, B., & Evans, D. Evaluating Differentially Private Machine Learning in Practice. USENIX Security, 2019.
