FRINK Linked Data Fragments server

Matches in SemOpenAlex for { <https://semopenalex.org/work/W2945336802> ?p ?o ?g. }

• W2945336802 abstract "Communication overhead is a major bottleneck hampering the scalability of distributed machine learning systems. Recently, there has been a surge of interest in using gradient compression to improve the communication efficiency of distributed neural network training. Using 1-bit quantization, signSGD with majority vote achieves a 32x reduction on communication cost. However, its convergence is based on unrealistic assumptions and can diverge in practice. In this paper, we propose a general distributed compressed SGD with Nesterov's momentum. We consider two-way compression, which compresses the gradients both to and from workers. Convergence analysis on nonconvex problems for general gradient compressors is provided. By partitioning the gradient into blocks, a blockwise compressor is introduced such that each gradient block is compressed and transmitted in 1-bit format with a scaling factor, leading to a nearly 32x reduction on communication. Experimental results show that the proposed method converges as fast as full-precision distributed momentum SGD and achieves the same testing accuracy. In particular, on distributed ResNet training with 7 workers on the ImageNet, the proposed algorithm achieves the same testing accuracy as momentum SGD using full-precision gradients, but with $46%$ less wall clock time." @default.
• W2945336802 created "2019-05-29" @default.
• W2945336802 creator A5042419001 @default.
• W2945336802 creator A5070273088 @default.
• W2945336802 creator A5072799172 @default.
• W2945336802 date "2019-05-26" @default.
• W2945336802 modified "2023-09-25" @default.
• W2945336802 title "Communication-Efficient Distributed Blockwise Momentum SGD with Error-Feedback" @default.
• W2945336802 cites W104184427 @default.
• W2945336802 cites W1591801644 @default.
• W2945336802 cites W1810943226 @default.
• W2945336802 cites W2102486516 @default.
• W2945336802 cites W2117539524 @default.
• W2945336802 cites W2127941149 @default.
• W2945336802 cites W2166706236 @default.
• W2945336802 cites W2168231600 @default.
• W2945336802 cites W2257979135 @default.
• W2945336802 cites W2302255633 @default.
• W2945336802 cites W2336650964 @default.
• W2945336802 cites W2405578611 @default.
• W2945336802 cites W2407022425 @default.
• W2945336802 cites W2579247884 @default.
• W2945336802 cites W2769644379 @default.
• W2945336802 cites W2890924858 @default.
• W2945336802 cites W2908007030 @default.
• W2945336802 cites W2911863041 @default.
• W2945336802 cites W2946815927 @default.
• W2945336802 cites W2948746540 @default.
• W2945336802 cites W2954365017 @default.
• W2945336802 cites W2963403868 @default.
• W2945336802 cites W2963540381 @default.
• W2945336802 cites W2963664311 @default.
• W2945336802 cites W2963803379 @default.
• W2945336802 cites W2964004663 @default.
• W2945336802 cites W2964163156 @default.
• W2945336802 cites W3101036738 @default.
• W2945336802 doi "https://doi.org/10.48550/arxiv.1905.10936" @default.
• W2945336802 hasPublicationYear "2019" @default.
• W2945336802 type Work @default.
• W2945336802 sameAs 2945336802 @default.
• W2945336802 citedByCount "1" @default.
• W2945336802 countsByYear W29453368022020 @default.
• W2945336802 crossrefType "posted-content" @default.
• W2945336802 hasAuthorship W2945336802A5042419001 @default.
• W2945336802 hasAuthorship W2945336802A5070273088 @default.
• W2945336802 hasAuthorship W2945336802A5072799172 @default.
• W2945336802 hasBestOaLocation W29453368021 @default.
• W2945336802 hasConcept C10138342 @default.
• W2945336802 hasConcept C111335779 @default.
• W2945336802 hasConcept C111919701 @default.
• W2945336802 hasConcept C113775141 @default.
• W2945336802 hasConcept C11413529 @default.
• W2945336802 hasConcept C149635348 @default.
• W2945336802 hasConcept C153258448 @default.
• W2945336802 hasConcept C154945302 @default.
• W2945336802 hasConcept C15744967 @default.
• W2945336802 hasConcept C162324750 @default.
• W2945336802 hasConcept C19417346 @default.
• W2945336802 hasConcept C2524010 @default.
• W2945336802 hasConcept C2777303404 @default.
• W2945336802 hasConcept C2779582901 @default.
• W2945336802 hasConcept C2779960059 @default.
• W2945336802 hasConcept C2780513914 @default.
• W2945336802 hasConcept C28855332 @default.
• W2945336802 hasConcept C33923547 @default.
• W2945336802 hasConcept C41008148 @default.
• W2945336802 hasConcept C48044578 @default.
• W2945336802 hasConcept C50522688 @default.
• W2945336802 hasConcept C50644808 @default.
• W2945336802 hasConcept C60718061 @default.
• W2945336802 hasConcept C77088390 @default.
• W2945336802 hasConceptScore W2945336802C10138342 @default.
• W2945336802 hasConceptScore W2945336802C111335779 @default.
• W2945336802 hasConceptScore W2945336802C111919701 @default.
• W2945336802 hasConceptScore W2945336802C113775141 @default.
• W2945336802 hasConceptScore W2945336802C11413529 @default.
• W2945336802 hasConceptScore W2945336802C149635348 @default.
• W2945336802 hasConceptScore W2945336802C153258448 @default.
• W2945336802 hasConceptScore W2945336802C154945302 @default.
• W2945336802 hasConceptScore W2945336802C15744967 @default.
• W2945336802 hasConceptScore W2945336802C162324750 @default.
• W2945336802 hasConceptScore W2945336802C19417346 @default.
• W2945336802 hasConceptScore W2945336802C2524010 @default.
• W2945336802 hasConceptScore W2945336802C2777303404 @default.
• W2945336802 hasConceptScore W2945336802C2779582901 @default.
• W2945336802 hasConceptScore W2945336802C2779960059 @default.
• W2945336802 hasConceptScore W2945336802C2780513914 @default.
• W2945336802 hasConceptScore W2945336802C28855332 @default.
• W2945336802 hasConceptScore W2945336802C33923547 @default.
• W2945336802 hasConceptScore W2945336802C41008148 @default.
• W2945336802 hasConceptScore W2945336802C48044578 @default.
• W2945336802 hasConceptScore W2945336802C50522688 @default.
• W2945336802 hasConceptScore W2945336802C50644808 @default.
• W2945336802 hasConceptScore W2945336802C60718061 @default.
• W2945336802 hasConceptScore W2945336802C77088390 @default.
• W2945336802 hasLocation W29453368021 @default.
• W2945336802 hasOpenAccess W2945336802 @default.
• W2945336802 hasPrimaryLocation W29453368021 @default.
• W2945336802 hasRelatedWork W2082482750 @default.
• W2945336802 hasRelatedWork W2087937280 @default.

• next