Matches in SemOpenAlex for { <https://semopenalex.org/work/W4288335809> ?p ?o ?g. }
Showing items 1 to 65 of
65
with 100 items per page.
- W4288335809 abstract "Communication bottleneck has been identified as a significant issue in distributed optimization of large-scale learning models. Recently, several approaches to mitigate this problem have been proposed, including different forms of gradient compression or computing local models and mixing them iteratively. In this paper, we propose emph{Qsparse-local-SGD} algorithm, which combines aggressive sparsification with quantization and local computation along with error compensation, by keeping track of the difference between the true and compressed gradients. We propose both synchronous and asynchronous implementations of emph{Qsparse-local-SGD}. We analyze convergence for emph{Qsparse-local-SGD} in the emph{distributed} setting for smooth non-convex and convex objective functions. We demonstrate that emph{Qsparse-local-SGD} converges at the same rate as vanilla distributed SGD for many important classes of sparsifiers and quantizers. We use emph{Qsparse-local-SGD} to train ResNet-50 on ImageNet and show that it results in significant savings over the state-of-the-art, in the number of bits transmitted to reach target accuracy." @default.
- W4288335809 created "2022-07-29" @default.
- W4288335809 creator A5020834141 @default.
- W4288335809 creator A5023777978 @default.
- W4288335809 creator A5074879219 @default.
- W4288335809 creator A5083980887 @default.
- W4288335809 date "2019-06-05" @default.
- W4288335809 modified "2023-09-29" @default.
- W4288335809 title "Qsparse-local-SGD: Distributed SGD with Quantization, Sparsification, and Local Computations" @default.
- W4288335809 doi "https://doi.org/10.48550/arxiv.1906.02367" @default.
- W4288335809 hasPublicationYear "2019" @default.
- W4288335809 type Work @default.
- W4288335809 citedByCount "0" @default.
- W4288335809 crossrefType "posted-content" @default.
- W4288335809 hasAuthorship W4288335809A5020834141 @default.
- W4288335809 hasAuthorship W4288335809A5023777978 @default.
- W4288335809 hasAuthorship W4288335809A5074879219 @default.
- W4288335809 hasAuthorship W4288335809A5083980887 @default.
- W4288335809 hasBestOaLocation W42883358091 @default.
- W4288335809 hasConcept C112680207 @default.
- W4288335809 hasConcept C11413529 @default.
- W4288335809 hasConcept C149635348 @default.
- W4288335809 hasConcept C151319957 @default.
- W4288335809 hasConcept C2524010 @default.
- W4288335809 hasConcept C26517878 @default.
- W4288335809 hasConcept C2780513914 @default.
- W4288335809 hasConcept C28855332 @default.
- W4288335809 hasConcept C31258907 @default.
- W4288335809 hasConcept C33923547 @default.
- W4288335809 hasConcept C38652104 @default.
- W4288335809 hasConcept C41008148 @default.
- W4288335809 hasConcept C45374587 @default.
- W4288335809 hasConcept C57869625 @default.
- W4288335809 hasConcept C80444323 @default.
- W4288335809 hasConceptScore W4288335809C112680207 @default.
- W4288335809 hasConceptScore W4288335809C11413529 @default.
- W4288335809 hasConceptScore W4288335809C149635348 @default.
- W4288335809 hasConceptScore W4288335809C151319957 @default.
- W4288335809 hasConceptScore W4288335809C2524010 @default.
- W4288335809 hasConceptScore W4288335809C26517878 @default.
- W4288335809 hasConceptScore W4288335809C2780513914 @default.
- W4288335809 hasConceptScore W4288335809C28855332 @default.
- W4288335809 hasConceptScore W4288335809C31258907 @default.
- W4288335809 hasConceptScore W4288335809C33923547 @default.
- W4288335809 hasConceptScore W4288335809C38652104 @default.
- W4288335809 hasConceptScore W4288335809C41008148 @default.
- W4288335809 hasConceptScore W4288335809C45374587 @default.
- W4288335809 hasConceptScore W4288335809C57869625 @default.
- W4288335809 hasConceptScore W4288335809C80444323 @default.
- W4288335809 hasLocation W42883358091 @default.
- W4288335809 hasOpenAccess W4288335809 @default.
- W4288335809 hasPrimaryLocation W42883358091 @default.
- W4288335809 hasRelatedWork W1481543858 @default.
- W4288335809 hasRelatedWork W1567314313 @default.
- W4288335809 hasRelatedWork W1965942711 @default.
- W4288335809 hasRelatedWork W2023074252 @default.
- W4288335809 hasRelatedWork W2040210784 @default.
- W4288335809 hasRelatedWork W2146084924 @default.
- W4288335809 hasRelatedWork W2358782222 @default.
- W4288335809 hasRelatedWork W2794719570 @default.
- W4288335809 hasRelatedWork W2914138317 @default.
- W4288335809 hasRelatedWork W3087324740 @default.
- W4288335809 isParatext "false" @default.
- W4288335809 isRetracted "false" @default.
- W4288335809 workType "article" @default.