"decentralized training" Papers

From Promise to Practice: Realizing High-performance Decentralized Training

Zesen Wang, Jiaojiao Zhang, Xuyang Wu et al.

Learning to Specialize: Joint Gating-Expert Training for Adaptive MoEs in Decentralized Settings

Yehya Farhat, Hamza ElMokhtar Shili, Fangshuo Liao et al.

Unveiling the Power of Multiple Gossip Steps: A Stability-Based Generalization Analysis in Decentralized Training

FedMef: Towards Memory-efficient Federated Dynamic Pruning

Hong Huang, Weiming Zhuang, Chen Chen et al.

Position: Exploring the Robustness of Pipeline-Parallelism-Based Decentralized Training

Lin Lu, Chenxi Dai, Wangcheng Tao et al.

Rethinking the Flat Minima Searching in Federated Learning

Taehwan Lee, Sung Whan Yoon