Tuesday 11 April, 2023 – 2.00-3.30pm (Online)
Speakers
– Aymeric Dieuleveut, Hi! PARIS Fellow @École polytechnique
– Constantin Philippenko, PhD @École polytechnique
Program
We will describe the Federated Learning context and the causes and types of heterogeneity. Then, we will study the impact of heterogeneity on learning: from distributed learning to FedAvg, and FedAvg to Scaffold. We will also relate to the case of learning with compression: for example why partial client participation and degrades convergence in the presence of heterogeneity. We will implement the basic methods on simple datasets.
The proposed paper presents the algorithm, which is one of the most critical algorithms from the point of view of heterogeneity and FL. Introduced in 2019, it had a great influence since.
Paper
– SCAFFOLD: Stochastic Controlled Averaging for Federated Learning, Sai Praneeth Karimireddy, Satyen Kale, Mehryar Mohri, Sashank Reddi, Sebastian Stich, Ananda Theertha Suresh Proceedings of the 37th International Conference on Machine Learning, PMLR 119:5132-5143, 2020.
Notebook
Download document