Matches in SemOpenAlex for { <https://semopenalex.org/work/W2989763744> ?p ?o ?g. }
Showing items 1 to 91 of
91
with 100 items per page.
- W2989763744 endingPage "13590" @default.
- W2989763744 startingPage "13579" @default.
- W2989763744 abstract "Synchronous mini-batch SGD is state-of-the-art for large-scale distributed machine learning. However, in practice, its convergence is bottlenecked by slow communication rounds between worker nodes. A natural solution to reduce communication is to use the emph{``local-SGD''} model in which the workers train their model independently and synchronize every once in a while. This algorithm improves the computation-communication trade-off but its convergence is not understood very well. We propose a non-asymptotic error analysis, which enables comparison to emph{one-shot averaging} i.e., a single communication round among independent workers, and emph{mini-batch averaging} i.e., communicating at every step. We also provide adaptive lower bounds on the communication frequency for large step-sizes ($ t^{-alpha} $, $ alphain (1/2 , 1 ) $) and show that emph{Local-SGD} reduces communication by a factor of $OBig(frac{sqrt{T}}{P^{3/2}}Big)$, with $T$ the total number of gradients and $P$ machines." @default.
- W2989763744 created "2019-12-05" @default.
- W2989763744 creator A5023396823 @default.
- W2989763744 creator A5056989506 @default.
- W2989763744 date "2019-01-01" @default.
- W2989763744 modified "2023-09-24" @default.
- W2989763744 title "Communication trade-offs for Local-SGD with large step size" @default.
- W2989763744 cites W1508912423 @default.
- W2989763744 cites W1516903196 @default.
- W2989763744 cites W1575244755 @default.
- W2989763744 cites W1780115997 @default.
- W2989763744 cites W1842094663 @default.
- W2989763744 cites W1877037013 @default.
- W2989763744 cites W1992208280 @default.
- W2989763744 cites W1999083556 @default.
- W2989763744 cites W2066623346 @default.
- W2989763744 cites W2072566913 @default.
- W2989763744 cites W2086161653 @default.
- W2989763744 cites W2091825929 @default.
- W2989763744 cites W2103971893 @default.
- W2989763744 cites W2112269233 @default.
- W2989763744 cites W2130062883 @default.
- W2989763744 cites W2138302120 @default.
- W2989763744 cites W2138855972 @default.
- W2989763744 cites W2144839971 @default.
- W2989763744 cites W2156779765 @default.
- W2989763744 cites W2159930037 @default.
- W2989763744 cites W2162287622 @default.
- W2989763744 cites W2166706236 @default.
- W2989763744 cites W2738275598 @default.
- W2989763744 cites W2755172545 @default.
- W2989763744 cites W2951781666 @default.
- W2989763744 cites W2963959597 @default.
- W2989763744 cites W2964198904 @default.
- W2989763744 cites W3103741939 @default.
- W2989763744 cites W59018853 @default.
- W2989763744 cites W88685657 @default.
- W2989763744 hasPublicationYear "2019" @default.
- W2989763744 type Work @default.
- W2989763744 sameAs 2989763744 @default.
- W2989763744 citedByCount "15" @default.
- W2989763744 countsByYear W29897637442019 @default.
- W2989763744 countsByYear W29897637442020 @default.
- W2989763744 countsByYear W29897637442021 @default.
- W2989763744 crossrefType "proceedings-article" @default.
- W2989763744 hasAuthorship W2989763744A5023396823 @default.
- W2989763744 hasAuthorship W2989763744A5056989506 @default.
- W2989763744 hasConcept C11413529 @default.
- W2989763744 hasConcept C162324750 @default.
- W2989763744 hasConcept C2777303404 @default.
- W2989763744 hasConcept C41008148 @default.
- W2989763744 hasConcept C45374587 @default.
- W2989763744 hasConcept C50522688 @default.
- W2989763744 hasConcept C80444323 @default.
- W2989763744 hasConceptScore W2989763744C11413529 @default.
- W2989763744 hasConceptScore W2989763744C162324750 @default.
- W2989763744 hasConceptScore W2989763744C2777303404 @default.
- W2989763744 hasConceptScore W2989763744C41008148 @default.
- W2989763744 hasConceptScore W2989763744C45374587 @default.
- W2989763744 hasConceptScore W2989763744C50522688 @default.
- W2989763744 hasConceptScore W2989763744C80444323 @default.
- W2989763744 hasLocation W29897637441 @default.
- W2989763744 hasOpenAccess W2989763744 @default.
- W2989763744 hasPrimaryLocation W29897637441 @default.
- W2989763744 hasRelatedWork W2108475251 @default.
- W2989763744 hasRelatedWork W2130062883 @default.
- W2989763744 hasRelatedWork W2153635508 @default.
- W2989763744 hasRelatedWork W2166706236 @default.
- W2989763744 hasRelatedWork W2535838896 @default.
- W2989763744 hasRelatedWork W2541884796 @default.
- W2989763744 hasRelatedWork W2888561381 @default.
- W2989763744 hasRelatedWork W2890924858 @default.
- W2989763744 hasRelatedWork W2900182564 @default.
- W2989763744 hasRelatedWork W2940453604 @default.
- W2989763744 hasRelatedWork W2944542720 @default.
- W2989763744 hasRelatedWork W2946270031 @default.
- W2989763744 hasRelatedWork W2963179579 @default.
- W2989763744 hasRelatedWork W2963803379 @default.
- W2989763744 hasRelatedWork W2970020383 @default.
- W2989763744 hasRelatedWork W2973096819 @default.
- W2989763744 hasRelatedWork W2981206218 @default.
- W2989763744 hasRelatedWork W2995022099 @default.
- W2989763744 hasRelatedWork W3037674069 @default.
- W2989763744 hasRelatedWork W3102816259 @default.
- W2989763744 hasVolume "32" @default.
- W2989763744 isParatext "false" @default.
- W2989763744 isRetracted "false" @default.
- W2989763744 magId "2989763744" @default.
- W2989763744 workType "article" @default.