FRINK Linked Data Fragments server

Matches in SemOpenAlex for { <https://semopenalex.org/work/W3136347532> ?p ?o ?g. }

• W3136347532 abstract "Synchronous Stochastic Gradient Descent (SGD) with data parallelism, the most popular parallel training strategy for deep learning, suffers from expensive gradient communications. Local SGD with periodic model averaging is a promising alternative to synchronous SGD. The algorithm allows each worker to locally update its own model, and periodically averages the model parameters across all the workers. While this algorithm enjoys less frequent communications, the convergence rate is strongly affected by the number of workers. In order to scale up the local SGD training without losing accuracy, the number of workers should be sufficiently small so that the model converges reasonably fast. In this paper, we discuss how to exploit the degree of parallelism in local SGD while maintaining model accuracy. Our training strategy employs multiple groups of processes and each group trains a local model based on data parallelism. The local models are periodically averaged across all the groups. Based on this hierarchical parallelism, we design a model averaging algorithm that has a cheaper communication cost than allreduce-based approach. We also propose a practical metric for finding the maximum number of workers that does not cause a significant accuracy loss. Our experimental results demonstrate that our proposed training strategy provides a significantly improved scalability while achieving a comparable model accuracy to synchronous SGD." @default.
• W3136347532 created "2021-03-29" @default.
• W3136347532 creator A5001525854 @default.
• W3136347532 creator A5013922013 @default.
• W3136347532 creator A5048248377 @default.
• W3136347532 creator A5074976770 @default.
• W3136347532 creator A5081888263 @default.
• W3136347532 date "2020-12-10" @default.
• W3136347532 modified "2023-10-18" @default.
• W3136347532 title "Communication-Efficient Local Stochastic Gradient Descent for Scalable Deep Learning" @default.
• W3136347532 cites W2059300917 @default.
• W3136347532 cites W2105549957 @default.
• W3136347532 cites W2131613942 @default.
• W3136347532 cites W2194775991 @default.
• W3136347532 cites W2263490141 @default.
• W3136347532 cites W2622263826 @default.
• W3136347532 cites W2739757502 @default.
• W3136347532 cites W2751713192 @default.
• W3136347532 cites W2787299084 @default.
• W3136347532 cites W2900182564 @default.
• W3136347532 cites W2904556356 @default.
• W3136347532 cites W2911342633 @default.
• W3136347532 cites W2963372104 @default.
• W3136347532 cites W2963655672 @default.
• W3136347532 cites W2963702144 @default.
• W3136347532 cites W2963959597 @default.
• W3136347532 cites W2964121744 @default.
• W3136347532 cites W2964243509 @default.
• W3136347532 cites W2970020383 @default.
• W3136347532 cites W2972163183 @default.
• W3136347532 cites W3037047862 @default.
• W3136347532 cites W3037875189 @default.
• W3136347532 cites W3118608800 @default.
• W3136347532 doi "https://doi.org/10.1109/bigdata50022.2020.9378178" @default.
• W3136347532 hasPublicationYear "2020" @default.
• W3136347532 type Work @default.
• W3136347532 sameAs 3136347532 @default.
• W3136347532 citedByCount "2" @default.
• W3136347532 countsByYear W31363475322021 @default.
• W3136347532 crossrefType "proceedings-article" @default.
• W3136347532 hasAuthorship W3136347532A5001525854 @default.
• W3136347532 hasAuthorship W3136347532A5013922013 @default.
• W3136347532 hasAuthorship W3136347532A5048248377 @default.
• W3136347532 hasAuthorship W3136347532A5074976770 @default.
• W3136347532 hasAuthorship W3136347532A5081888263 @default.
• W3136347532 hasConcept C108583219 @default.
• W3136347532 hasConcept C11413529 @default.
• W3136347532 hasConcept C121332964 @default.
• W3136347532 hasConcept C141934464 @default.
• W3136347532 hasConcept C154945302 @default.
• W3136347532 hasConcept C162324750 @default.
• W3136347532 hasConcept C165696696 @default.
• W3136347532 hasConcept C173608175 @default.
• W3136347532 hasConcept C176217482 @default.
• W3136347532 hasConcept C206688291 @default.
• W3136347532 hasConcept C21547014 @default.
• W3136347532 hasConcept C2777303404 @default.
• W3136347532 hasConcept C2778755073 @default.
• W3136347532 hasConcept C2781172179 @default.
• W3136347532 hasConcept C38652104 @default.
• W3136347532 hasConcept C41008148 @default.
• W3136347532 hasConcept C48044578 @default.
• W3136347532 hasConcept C50522688 @default.
• W3136347532 hasConcept C50644808 @default.
• W3136347532 hasConcept C61483411 @default.
• W3136347532 hasConcept C62520636 @default.
• W3136347532 hasConcept C77088390 @default.
• W3136347532 hasConceptScore W3136347532C108583219 @default.
• W3136347532 hasConceptScore W3136347532C11413529 @default.
• W3136347532 hasConceptScore W3136347532C121332964 @default.
• W3136347532 hasConceptScore W3136347532C141934464 @default.
• W3136347532 hasConceptScore W3136347532C154945302 @default.
• W3136347532 hasConceptScore W3136347532C162324750 @default.
• W3136347532 hasConceptScore W3136347532C165696696 @default.
• W3136347532 hasConceptScore W3136347532C173608175 @default.
• W3136347532 hasConceptScore W3136347532C176217482 @default.
• W3136347532 hasConceptScore W3136347532C206688291 @default.
• W3136347532 hasConceptScore W3136347532C21547014 @default.
• W3136347532 hasConceptScore W3136347532C2777303404 @default.
• W3136347532 hasConceptScore W3136347532C2778755073 @default.
• W3136347532 hasConceptScore W3136347532C2781172179 @default.
• W3136347532 hasConceptScore W3136347532C38652104 @default.
• W3136347532 hasConceptScore W3136347532C41008148 @default.
• W3136347532 hasConceptScore W3136347532C48044578 @default.
• W3136347532 hasConceptScore W3136347532C50522688 @default.
• W3136347532 hasConceptScore W3136347532C50644808 @default.
• W3136347532 hasConceptScore W3136347532C61483411 @default.
• W3136347532 hasConceptScore W3136347532C62520636 @default.
• W3136347532 hasConceptScore W3136347532C77088390 @default.
• W3136347532 hasFunder F4320306084 @default.
• W3136347532 hasLocation W31363475321 @default.
• W3136347532 hasOpenAccess W3136347532 @default.
• W3136347532 hasPrimaryLocation W31363475321 @default.
• W3136347532 hasRelatedWork W11910490 @default.
• W3136347532 hasRelatedWork W1229628 @default.
• W3136347532 hasRelatedWork W13067065 @default.
• W3136347532 hasRelatedWork W13846533 @default.
• W3136347532 hasRelatedWork W2700343 @default.
• W3136347532 hasRelatedWork W3422034 @default.
• W3136347532 hasRelatedWork W5143923 @default.

• next