Bandwidth-Efficient Federated Learning: A Preprocessing and Compression-based Approach using MNIST

Abstract: Federated Learning (FL) allows training a multiclient model without centralizing raw data, which makes it an
excellent choice to apply on privacy-sensitive tasks such as digit
recognition using the MNIST dataset. But the issue is non-IID
data distributions and communications overhead are a major
challenge that can adversely impact model accuracy and
efficiency. The study introduces a streamlined FL framework
with the addition of data preprocessing, proximal
regularization (FedProx), gradient quantization and top-k
sparsification. CNN was trained over 100 clients under IID and
non-IID (Dirichlet =80alpha:0.5) partitioning with rounds
capped at 10. Experimental results demonstrate that whereas
baseline FedAvg obtained 98.2% accuracy on IID data, 85.4%
on non-IID data, the proposed optimizations have the
significant contributions to stability and efficiency. In
particular and specifically, FedProx boosted non-IID accuracy
to 88.6 percent, 8-bit quantization reduced bandwidth by 75
percent at the cost of less than one percent, and top-10%
sparsification conserved 90 percent of communication
incurring only ~3 percent accuracy loss. In addition, FedProx
combined with the gradient compression provided a practical
trade-off between accuracy and communication efficiency, and
performed better than the baseline methods. These results
show that proximal-regularized communication-efficient FL
can address non-IID hurdles and make federated training
resource-aware and scalable without considerable performance
loss.