Matches in SemOpenAlex for { <https://semopenalex.org/work/W3100770245> ?p ?o ?g. }
Showing items 1 to 94 of
94
with 100 items per page.
- W3100770245 endingPage "20900" @default.
- W3100770245 startingPage "20889" @default.
- W3100770245 abstract "In this paper, we propose a unified analysis of variants of distributed SGD with arbitrary compressions and delayed updates. Our framework is general enough to cover different variants of quantized SGD, Error-Compensated SGD (EC-SGD) and SGD with delayed updates (D-SGD). Via a single theorem, we derive the complexity results for all the methods that fit our framework. For the existing methods, this theorem gives the best-known complexity results. Moreover, using our general scheme, we develop new variants of SGD that combine variance reduction or arbitrary sampling with error feedback and quantization and derive the convergence rates for these methods beating the state-of-the-art results. In order to illustrate the strength of our framework, we develop 16 new methods that fit this. In particular, we propose the first method called EC-SGD-DIANA that is based on error-feedback for biased compression operator and quantization of gradient differences and prove the convergence guarantees showing that EC-SGD-DIANA converges to the exact optimum asymptotically in expectation with constant learning rate for both convex and strongly convex objectives when workers compute full gradients of their loss functions. Moreover, for the case when the loss function of the worker has the form of finite sum, we modified the method and got a new one called EC-LSVRG-DIANA which is the first distributed stochastic method with error feedback and variance reduction that converges to the exact optimum asymptotically in expectation with a constant learning rate." @default.
- W3100770245 created "2020-11-23" @default.
- W3100770245 creator A5017429728 @default.
- W3100770245 creator A5035028359 @default.
- W3100770245 creator A5036598221 @default.
- W3100770245 creator A5087594198 @default.
- W3100770245 date "2020-01-01" @default.
- W3100770245 modified "2023-09-26" @default.
- W3100770245 title "Linearly Converging Error Compensated SGD" @default.
- W3100770245 hasPublicationYear "2020" @default.
- W3100770245 type Work @default.
- W3100770245 sameAs 3100770245 @default.
- W3100770245 citedByCount "19" @default.
- W3100770245 countsByYear W31007702452020 @default.
- W3100770245 countsByYear W31007702452021 @default.
- W3100770245 crossrefType "proceedings-article" @default.
- W3100770245 hasAuthorship W3100770245A5017429728 @default.
- W3100770245 hasAuthorship W3100770245A5035028359 @default.
- W3100770245 hasAuthorship W3100770245A5036598221 @default.
- W3100770245 hasAuthorship W3100770245A5087594198 @default.
- W3100770245 hasConcept C105795698 @default.
- W3100770245 hasConcept C111335779 @default.
- W3100770245 hasConcept C112680207 @default.
- W3100770245 hasConcept C11413529 @default.
- W3100770245 hasConcept C126255220 @default.
- W3100770245 hasConcept C127162648 @default.
- W3100770245 hasConcept C145446738 @default.
- W3100770245 hasConcept C162324750 @default.
- W3100770245 hasConcept C181789720 @default.
- W3100770245 hasConcept C19499675 @default.
- W3100770245 hasConcept C199360897 @default.
- W3100770245 hasConcept C2524010 @default.
- W3100770245 hasConcept C2777027219 @default.
- W3100770245 hasConcept C2777303404 @default.
- W3100770245 hasConcept C28826006 @default.
- W3100770245 hasConcept C28855332 @default.
- W3100770245 hasConcept C31258907 @default.
- W3100770245 hasConcept C33923547 @default.
- W3100770245 hasConcept C41008148 @default.
- W3100770245 hasConcept C50522688 @default.
- W3100770245 hasConcept C57869625 @default.
- W3100770245 hasConcept C62644790 @default.
- W3100770245 hasConceptScore W3100770245C105795698 @default.
- W3100770245 hasConceptScore W3100770245C111335779 @default.
- W3100770245 hasConceptScore W3100770245C112680207 @default.
- W3100770245 hasConceptScore W3100770245C11413529 @default.
- W3100770245 hasConceptScore W3100770245C126255220 @default.
- W3100770245 hasConceptScore W3100770245C127162648 @default.
- W3100770245 hasConceptScore W3100770245C145446738 @default.
- W3100770245 hasConceptScore W3100770245C162324750 @default.
- W3100770245 hasConceptScore W3100770245C181789720 @default.
- W3100770245 hasConceptScore W3100770245C19499675 @default.
- W3100770245 hasConceptScore W3100770245C199360897 @default.
- W3100770245 hasConceptScore W3100770245C2524010 @default.
- W3100770245 hasConceptScore W3100770245C2777027219 @default.
- W3100770245 hasConceptScore W3100770245C2777303404 @default.
- W3100770245 hasConceptScore W3100770245C28826006 @default.
- W3100770245 hasConceptScore W3100770245C28855332 @default.
- W3100770245 hasConceptScore W3100770245C31258907 @default.
- W3100770245 hasConceptScore W3100770245C33923547 @default.
- W3100770245 hasConceptScore W3100770245C41008148 @default.
- W3100770245 hasConceptScore W3100770245C50522688 @default.
- W3100770245 hasConceptScore W3100770245C57869625 @default.
- W3100770245 hasConceptScore W3100770245C62644790 @default.
- W3100770245 hasLocation W31007702451 @default.
- W3100770245 hasOpenAccess W3100770245 @default.
- W3100770245 hasPrimaryLocation W31007702451 @default.
- W3100770245 hasRelatedWork W2153635508 @default.
- W3100770245 hasRelatedWork W2194775991 @default.
- W3100770245 hasRelatedWork W2407022425 @default.
- W3100770245 hasRelatedWork W2535838896 @default.
- W3100770245 hasRelatedWork W2541884796 @default.
- W3100770245 hasRelatedWork W2769644379 @default.
- W3100770245 hasRelatedWork W2808495300 @default.
- W3100770245 hasRelatedWork W2890924858 @default.
- W3100770245 hasRelatedWork W2911863041 @default.
- W3100770245 hasRelatedWork W2914138317 @default.
- W3100770245 hasRelatedWork W2935990182 @default.
- W3100770245 hasRelatedWork W2963766684 @default.
- W3100770245 hasRelatedWork W2970289928 @default.
- W3100770245 hasRelatedWork W2971064744 @default.
- W3100770245 hasRelatedWork W2973096819 @default.
- W3100770245 hasRelatedWork W2981302852 @default.
- W3100770245 hasRelatedWork W3007528421 @default.
- W3100770245 hasRelatedWork W3093577692 @default.
- W3100770245 hasRelatedWork W3113984671 @default.
- W3100770245 hasRelatedWork W3089975361 @default.
- W3100770245 hasVolume "33" @default.
- W3100770245 isParatext "false" @default.
- W3100770245 isRetracted "false" @default.
- W3100770245 magId "3100770245" @default.
- W3100770245 workType "article" @default.