Matches in SemOpenAlex for { <https://semopenalex.org/work/W2996442797> ?p ?o ?g. }
- W2996442797 abstract "To accelerate the training of machine learning models, distributed stochastic gradient descent (SGD) and its variants have been widely adopted, which apply multiple workers in parallel to speed up training. Among them, Local SGD has gained much attention due to its lower communication cost. Nevertheless, when the data distribution on workers is non-identical, Local SGD requires $O(T^{frac{3}{4}} N^{frac{3}{4}})$ communications to maintain its emph{linear iteration speedup} property, where $T$ is the total number of iterations and $N$ is the number of workers. In this paper, we propose Variance Reduced Local SGD (VRL-SGD) to further reduce the communication complexity. Benefiting from eliminating the dependency on the gradient variance among workers, we theoretically prove that VRL-SGD achieves a emph{linear iteration speedup} with a lower communication complexity $O(T^{frac{1}{2}} N^{frac{3}{2}})$ even if workers access non-identical datasets. We conduct experiments on three machine learning tasks, and the experimental results demonstrate that VRL-SGD performs impressively better than Local SGD when the data among workers are quite diverse." @default.
- W2996442797 created "2019-12-26" @default.
- W2996442797 creator A5007825653 @default.
- W2996442797 creator A5008272052 @default.
- W2996442797 creator A5009264739 @default.
- W2996442797 creator A5048237545 @default.
- W2996442797 creator A5054743312 @default.
- W2996442797 creator A5072210488 @default.
- W2996442797 date "2019-09-25" @default.
- W2996442797 modified "2023-09-26" @default.
- W2996442797 title "Variance Reduced Local SGD with Lower Communication Complexity." @default.
- W2996442797 cites W1552847225 @default.
- W2996442797 cites W1571416372 @default.
- W2996442797 cites W1779452081 @default.
- W2996442797 cites W1832693441 @default.
- W2996442797 cites W1836465849 @default.
- W2996442797 cites W2107438106 @default.
- W2996442797 cites W2108598243 @default.
- W2996442797 cites W2130062883 @default.
- W2996442797 cites W2135482703 @default.
- W2996442797 cites W2183341477 @default.
- W2996442797 cites W2216483973 @default.
- W2996442797 cites W2250539671 @default.
- W2996442797 cites W2535838896 @default.
- W2996442797 cites W2769644379 @default.
- W2996442797 cites W2789243870 @default.
- W2996442797 cites W2888206291 @default.
- W2996442797 cites W2888561381 @default.
- W2996442797 cites W2899771611 @default.
- W2996442797 cites W2900182564 @default.
- W2996442797 cites W2908007030 @default.
- W2996442797 cites W2911863041 @default.
- W2996442797 cites W2944542720 @default.
- W2996442797 cites W2946270031 @default.
- W2996442797 cites W2946815927 @default.
- W2996442797 cites W2954365017 @default.
- W2996442797 cites W2955229517 @default.
- W2996442797 cites W2962696654 @default.
- W2996442797 cites W2963179579 @default.
- W2996442797 cites W2963470657 @default.
- W2996442797 cites W2963803379 @default.
- W2996442797 cites W2963804082 @default.
- W2996442797 cites W2995022099 @default.
- W2996442797 cites W3101036738 @default.
- W2996442797 cites W3102762242 @default.
- W2996442797 cites W3102816259 @default.
- W2996442797 cites W855165587 @default.
- W2996442797 hasPublicationYear "2019" @default.
- W2996442797 type Work @default.
- W2996442797 sameAs 2996442797 @default.
- W2996442797 citedByCount "39" @default.
- W2996442797 countsByYear W29964427972019 @default.
- W2996442797 countsByYear W29964427972020 @default.
- W2996442797 countsByYear W29964427972021 @default.
- W2996442797 crossrefType "posted-content" @default.
- W2996442797 hasAuthorship W2996442797A5007825653 @default.
- W2996442797 hasAuthorship W2996442797A5008272052 @default.
- W2996442797 hasAuthorship W2996442797A5009264739 @default.
- W2996442797 hasAuthorship W2996442797A5048237545 @default.
- W2996442797 hasAuthorship W2996442797A5054743312 @default.
- W2996442797 hasAuthorship W2996442797A5072210488 @default.
- W2996442797 hasConcept C11413529 @default.
- W2996442797 hasConcept C119857082 @default.
- W2996442797 hasConcept C121955636 @default.
- W2996442797 hasConcept C144133560 @default.
- W2996442797 hasConcept C154945302 @default.
- W2996442797 hasConcept C173608175 @default.
- W2996442797 hasConcept C179145077 @default.
- W2996442797 hasConcept C179799912 @default.
- W2996442797 hasConcept C196083921 @default.
- W2996442797 hasConcept C206688291 @default.
- W2996442797 hasConcept C33923547 @default.
- W2996442797 hasConcept C41008148 @default.
- W2996442797 hasConcept C50644808 @default.
- W2996442797 hasConcept C68339613 @default.
- W2996442797 hasConcept C80444323 @default.
- W2996442797 hasConceptScore W2996442797C11413529 @default.
- W2996442797 hasConceptScore W2996442797C119857082 @default.
- W2996442797 hasConceptScore W2996442797C121955636 @default.
- W2996442797 hasConceptScore W2996442797C144133560 @default.
- W2996442797 hasConceptScore W2996442797C154945302 @default.
- W2996442797 hasConceptScore W2996442797C173608175 @default.
- W2996442797 hasConceptScore W2996442797C179145077 @default.
- W2996442797 hasConceptScore W2996442797C179799912 @default.
- W2996442797 hasConceptScore W2996442797C196083921 @default.
- W2996442797 hasConceptScore W2996442797C206688291 @default.
- W2996442797 hasConceptScore W2996442797C33923547 @default.
- W2996442797 hasConceptScore W2996442797C41008148 @default.
- W2996442797 hasConceptScore W2996442797C50644808 @default.
- W2996442797 hasConceptScore W2996442797C68339613 @default.
- W2996442797 hasConceptScore W2996442797C80444323 @default.
- W2996442797 hasOpenAccess W2996442797 @default.
- W2996442797 hasRelatedWork W2194775991 @default.
- W2996442797 hasRelatedWork W2535838896 @default.
- W2996442797 hasRelatedWork W2541884796 @default.
- W2996442797 hasRelatedWork W2807006176 @default.
- W2996442797 hasRelatedWork W2888561381 @default.
- W2996442797 hasRelatedWork W2900182564 @default.
- W2996442797 hasRelatedWork W2963179579 @default.
- W2996442797 hasRelatedWork W2963300197 @default.