Matches in SemOpenAlex for { <https://semopenalex.org/work/W3130710521> ?p ?o ?g. }
Showing items 1 to 87 of
87
with 100 items per page.
- W3130710521 abstract "The heavy communication for model synchronization is a major bottleneck for scaling up the distributed deep neural network training to many workers. Moreover, model synchronization can suffer from long delays in scenarios such as federated learning and geo-distributed training. Thus, it is crucial that the distributed training methods are both textit{delay-tolerant} AND textit{communication-efficient}. However, existing works cannot simultaneously address the communication delay and bandwidth constraint. To address this important and challenging problem, we propose a novel training framework OLCOtextsubscript{3} to achieve delay tolerance with a low communication budget by using stale information. OLCOtextsubscript{3} introduces novel staleness compensation and compression compensation to combat the influence of staleness and compression error. Theoretical analysis shows that OLCOtextsubscript{3} achieves the same sub-linear convergence rate as the vanilla synchronous stochastic gradient descent (SGD) method. Extensive experiments on deep learning tasks verify the effectiveness of OLCOtextsubscript{3} and its advantages over existing works." @default.
- W3130710521 created "2021-03-01" @default.
- W3130710521 creator A5014924409 @default.
- W3130710521 creator A5027255721 @default.
- W3130710521 creator A5060016795 @default.
- W3130710521 creator A5063099787 @default.
- W3130710521 date "2021-05-04" @default.
- W3130710521 modified "2023-09-23" @default.
- W3130710521 title "Delay-Tolerant Local SGD for Efficient Distributed Training" @default.
- W3130710521 hasPublicationYear "2021" @default.
- W3130710521 type Work @default.
- W3130710521 sameAs 3130710521 @default.
- W3130710521 citedByCount "0" @default.
- W3130710521 crossrefType "journal-article" @default.
- W3130710521 hasAuthorship W3130710521A5014924409 @default.
- W3130710521 hasAuthorship W3130710521A5027255721 @default.
- W3130710521 hasAuthorship W3130710521A5060016795 @default.
- W3130710521 hasAuthorship W3130710521A5063099787 @default.
- W3130710521 hasConcept C120314980 @default.
- W3130710521 hasConcept C121332964 @default.
- W3130710521 hasConcept C127162648 @default.
- W3130710521 hasConcept C149635348 @default.
- W3130710521 hasConcept C153294291 @default.
- W3130710521 hasConcept C154945302 @default.
- W3130710521 hasConcept C15744967 @default.
- W3130710521 hasConcept C162324750 @default.
- W3130710521 hasConcept C19417346 @default.
- W3130710521 hasConcept C206688291 @default.
- W3130710521 hasConcept C2776257435 @default.
- W3130710521 hasConcept C2777211547 @default.
- W3130710521 hasConcept C2777303404 @default.
- W3130710521 hasConcept C2778562939 @default.
- W3130710521 hasConcept C2779582901 @default.
- W3130710521 hasConcept C2780513914 @default.
- W3130710521 hasConcept C2984842247 @default.
- W3130710521 hasConcept C31258907 @default.
- W3130710521 hasConcept C41008148 @default.
- W3130710521 hasConcept C50522688 @default.
- W3130710521 hasConcept C50644808 @default.
- W3130710521 hasConceptScore W3130710521C120314980 @default.
- W3130710521 hasConceptScore W3130710521C121332964 @default.
- W3130710521 hasConceptScore W3130710521C127162648 @default.
- W3130710521 hasConceptScore W3130710521C149635348 @default.
- W3130710521 hasConceptScore W3130710521C153294291 @default.
- W3130710521 hasConceptScore W3130710521C154945302 @default.
- W3130710521 hasConceptScore W3130710521C15744967 @default.
- W3130710521 hasConceptScore W3130710521C162324750 @default.
- W3130710521 hasConceptScore W3130710521C19417346 @default.
- W3130710521 hasConceptScore W3130710521C206688291 @default.
- W3130710521 hasConceptScore W3130710521C2776257435 @default.
- W3130710521 hasConceptScore W3130710521C2777211547 @default.
- W3130710521 hasConceptScore W3130710521C2777303404 @default.
- W3130710521 hasConceptScore W3130710521C2778562939 @default.
- W3130710521 hasConceptScore W3130710521C2779582901 @default.
- W3130710521 hasConceptScore W3130710521C2780513914 @default.
- W3130710521 hasConceptScore W3130710521C2984842247 @default.
- W3130710521 hasConceptScore W3130710521C31258907 @default.
- W3130710521 hasConceptScore W3130710521C41008148 @default.
- W3130710521 hasConceptScore W3130710521C50522688 @default.
- W3130710521 hasConceptScore W3130710521C50644808 @default.
- W3130710521 hasLocation W31307105211 @default.
- W3130710521 hasOpenAccess W3130710521 @default.
- W3130710521 hasPrimaryLocation W31307105211 @default.
- W3130710521 hasRelatedWork W1569963871 @default.
- W3130710521 hasRelatedWork W1603875907 @default.
- W3130710521 hasRelatedWork W2351462273 @default.
- W3130710521 hasRelatedWork W2963889845 @default.
- W3130710521 hasRelatedWork W2981289008 @default.
- W3130710521 hasRelatedWork W3007639695 @default.
- W3130710521 hasRelatedWork W3014950839 @default.
- W3130710521 hasRelatedWork W3045665711 @default.
- W3130710521 hasRelatedWork W3109549377 @default.
- W3130710521 hasRelatedWork W3119706528 @default.
- W3130710521 hasRelatedWork W3128855217 @default.
- W3130710521 hasRelatedWork W3138409184 @default.
- W3130710521 hasRelatedWork W3164046003 @default.
- W3130710521 hasRelatedWork W3166992878 @default.
- W3130710521 hasRelatedWork W3183596499 @default.
- W3130710521 hasRelatedWork W3187461605 @default.
- W3130710521 hasRelatedWork W3192649393 @default.
- W3130710521 hasRelatedWork W3194022000 @default.
- W3130710521 hasRelatedWork W3202331934 @default.
- W3130710521 hasRelatedWork W3208314527 @default.
- W3130710521 isParatext "false" @default.
- W3130710521 isRetracted "false" @default.
- W3130710521 magId "3130710521" @default.
- W3130710521 workType "article" @default.