GitHub topics: partial-allreduce
Shigangli/eager-SGD
Eager-SGD is a decentralized asynchronous SGD. It utilizes novel partial collectives operations to accumulate the gradients across all the processes.
Language: Python - Size: 1.31 MB - Last synced at: over 1 year ago - Pushed at: over 3 years ago - Stars: 7 - Forks: 0
Shigangli/WAGMA-SGD
WAGMA-SGD is a decentralized asynchronous SGD based on wait-avoiding group model averaging. The synchronization is relaxed by making the collectives externally-triggerable, namely, a collective can be initiated without requiring that all the processes enter it. It partially reduces the data within non-overlapping groups of process, improving the parallel scalability.
Language: Python - Size: 1.11 MB - Last synced at: over 2 years ago - Pushed at: almost 4 years ago - Stars: 6 - Forks: 0