Topic: "gradient-compression"
synxlin/deep-gradient-compression
[ICLR 2018] Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training
Language: Python - Size: 316 KB - Last synced at: 11 months ago - Pushed at: 11 months ago - Stars: 207 - Forks: 45
AI-secure/DataLens
[CCS 2021] "DataLens: Scalable Privacy Preserving Training via Gradient Compression and Aggregation" by Boxin Wang*, Fan Wu*, Yunhui Long*, Luka Rimanic, Ce Zhang, Bo Li
Language: Python - Size: 69.3 KB - Last synced at: over 1 year ago - Pushed at: over 3 years ago - Stars: 34 - Forks: 7
xinyandai/gradient-quantization
vector quantization for stochastic gradient descent.
Language: Python - Size: 111 MB - Last synced at: over 2 years ago - Pushed at: about 5 years ago - Stars: 31 - Forks: 7
HaoweiLi778/JointSQ
Simple Implementation of the CVPR 2024 Paper "JointSQ: Joint Sparsification-Quantization for Distributed Learning"
Language: Python - Size: 405 KB - Last synced at: 5 months ago - Pushed at: 5 months ago - Stars: 10 - Forks: 0
vineeths96/Gradient-Compression
We present a set of all-reduce compatible gradient compression algorithms which significantly reduce the communication overhead while maintaining the performance of vanilla SGD. We empirically evaluate the performance of the compression methods by training deep neural networks on the CIFAR10 dataset.
Language: Python - Size: 32.5 MB - Last synced at: about 2 years ago - Pushed at: over 3 years ago - Stars: 5 - Forks: 3
xinyandai/mpi-tensorflow
😂Distributed optimizer implemented with TensorFlow MPI operation
Language: Python - Size: 15.8 MB - Last synced at: over 2 years ago - Pushed at: about 6 years ago - Stars: 2 - Forks: 1
anishacharya/BGMD-AISTATS-2022
Geometric median (GM) is a classical method in statistics for achieving a robust estimation of the uncorrupted data; under gross corruption, it achieves the optimal breakdown point of 0.5. However, its computational complexity makes it infeasible for robustifying stochastic gradient descent (SGD) for high-dimensional optimization problems. In this paper, we show that by applying Gm to only a judiciously chosen block of coordinates at a time and using a memory mechanism, one can retain the breakdown point of 0.5 for smooth non-convex problems, with non-asymptotic convergence rates comparable to the SGD with GM.
Language: Python - Size: 8.92 MB - Last synced at: over 2 years ago - Pushed at: over 2 years ago - Stars: 1 - Forks: 0
