Model-Contrastive Federated Learning
2021
1,109 citations
Abstract
Federated learning enables multiple parties to collaboratively train a machine learning model without communicating their local data. A key challenge in federated learning is to handle the heterogeneity of local data distribution across parties. Although many studies have been proposed to address this challenge, we find that they fail to achieve high performance in image datasets with deep learning models. In this paper, we propose MOON: model-contrastive federated learning. MOON is a simple and effective federated learning framework. The key idea of MOON is to utilize the similarity between model representations to correct the local training of individual parties, i.e., conducting contrastive learning in model-level. Our extensive experiments show that MOON significantly outperforms the other state-of-the-art federated learning algorithms on various image classification tasks.
Keywords
Computer scienceFederated learningKey (lock)Similarity (geometry)Artificial intelligenceMachine learningDeep learningImage (mathematics)
Affiliated Institutions
Related Publications
Federated Learning with Non-IID Data
Federated learning enables resource-constrained edge compute devices, such as mobile phones and IoT devices, to learn a shared model for prediction, while keeping the training d...
Advances and Open Problems in Federated Learning
Federated learning (FL) is a machine learning setting where many clients (e.g. mobile devices or whole organizations) collaboratively train a model under the orchestration of a ...
Pre-Trained Image Processing Transformer
As the computing power of modern hardware is increasing strongly, pre-trained deep learning models (e.g., BERT, GPT-3) learned on large-scale datasets have shown their effective...
Federated Learning With Differential Privacy: Algorithms and Performance Analysis
Federated learning (FL), as a type of distributed machine learning, is capable of significantly preserving clients’ private data from being exposed to adversaries. Ne...
The Unreasonable Effectiveness of Deep Features as a Perceptual Metric
While it is nearly effortless for humans to quickly assess the perceptual similarity between two images, the underlying processes are thought to be quite complex. Despite this, ...
Publication Info
- Year
- 2021
- Type
- article
- Pages
- 10708-10717
- Citations
- 1109
- Access
- Closed
External Links
Social Impact
Altmetric
PlumX Metrics
Social media, news, blog, policy document mentions
Citation Metrics
1109
OpenAlex
Cite This
Qinbin Li,
Bingsheng He,
Dawn Song
(2021).
Model-Contrastive Federated Learning.
, 10708-10717.
https://doi.org/10.1109/cvpr46437.2021.01057
Identifiers
- DOI
- 10.1109/cvpr46437.2021.01057