As an enabler of sixth-generation communication technology (6G), Federated Learning (FL) triggers a paradigm shift from "connected things" to "connected intelligence". FL implements on-device learning, where massive end devices jointly and locally train a model without private data leakage. However, FL suffers from problems of low accuracy and convergence rate when no data is shared to the central server and the data distribution is non-IID. In recent years, attempts have been made on hybrid FL, where very small amounts of data (e.g., less than 1%) is shared from the participants. With the opportunities brought by shared data, we notice that the server is capable of receiving the data in order to assist the FL process and mitigate the challenge of non-IID. Notably, existing hybrid FL only applies the model-level technologies belonging to the traditional FL and does not make full use of the characteristics of shared data to make targeted improvements. In this paper, we propose FedAux, a novel hybrid FL method at knowledge-level, which utilizes shared data to construct an auxiliary model and then transfer general knowledge to traditional aggregated model or client model for enhancing the accuracy of global model and speeding up the convergence of global model. We also propose two specific knowledge transfer strategies named c-transfer and i-transfer. We conduct extensive analysis and evaluation of our methods against the well-known FL methods, FedAvg and Hybrid-FL protocol. The results indicate that FedAux shows higher accuracy (10.89%) and faster convergence rate compared with other methods.
|Title of host publication||ICC 2022 - IEEE International Conference on Communications|
|Number of pages||6|
|Publication status||Published - 11 Aug 2022|
|Event|| IEEE International Conference on Communications - |
Duration: 16 May 2022 → 20 May 2022
|Name||IEEE International Conference on Communications|
|Conference||IEEE International Conference on Communications|
|Abbreviated title||ICC 2022|
|Period||16/05/22 → 20/05/22|
Bibliographical noteFunding Information:
This work was partially supported by the National Key R&D Program of China(2019YFB1703901), National Science Fund for Distinguished Young Scholars(62025205), and the National Natural Science Foundation of China (No. 62032020, 62002292, 62102317), the Natural Science Basic Research Plan in Shaanxi Province of China (2020JQ-207) and the China Postdoctoral Science Foundation (No. 2021M702671).
© 2022 IEEE.
- 6G networks
- Federated learning
- Non-IID data
- Feature transferable theory