FRINK Linked Data Fragments server

Matches in SemOpenAlex for { <https://semopenalex.org/work/W3204244704> ?p ?o ?g. }

• W3204244704 abstract "Communication overhead is the key challenge for distributed training. Gradient compression is a widely used approach to reduce communication traffic. When combining with a parallel communication mechanism method like pipeline, gradient compression technique can greatly alleviate the impact of communication overhead. However, there exist two problems of gradient compression technique to be solved. Firstly, gradient compression brings in extra computation cost, which will delay the next training iteration. Secondly, gradient compression usually leads to a decrease in convergence accuracy. In this paper, we combine parallel mechanism with gradient quantization and delayed full-gradient compensation, and propose a new distributed optimization method named CD-SGD, which can hide the overhead of gradient compression, overlap part of the communication and obtain high convergence accuracy. The local update operation in CD-SGD allows the next iteration to be launched quickly without waiting for the completion of gradient compression and the current communication process. Besides, the accuracy loss caused by gradient compression is solved by k-step correction method introduced in CD-SGD. We prove that CD-SGD has convergence guarantee and it achieves at least convergence rate. We conduct extensive experiments on MXNet to verify the convergence properties and scaling performance of CD-SGD. Experimental results on a 16-GPU cluster show that convergence accuracy of CD-SGD is close to or even slightly better than that of S-SGD, and its end-to-end time is 30 less than 2-bit gradient compression under a 56Gbps bandwidth environment." @default.
• W3204244704 created "2021-10-11" @default.
• W3204244704 creator A5000688713 @default.
• W3204244704 creator A5006729432 @default.
• W3204244704 creator A5007572801 @default.
• W3204244704 creator A5042353509 @default.
• W3204244704 creator A5080680105 @default.
• W3204244704 date "2021-08-09" @default.
• W3204244704 modified "2023-10-16" @default.
• W3204244704 title "CD-SGD: Distributed Stochastic Gradient Descent with Compression and Delay Compensation" @default.
• W3204244704 cites W2108598243 @default.
• W3204244704 cites W2405578611 @default.
• W3204244704 cites W2407022425 @default.
• W3204244704 cites W2580688187 @default.
• W3204244704 cites W2904556356 @default.
• W3204244704 cites W2938693278 @default.
• W3204244704 cites W2963786636 @default.
• W3204244704 cites W2967558351 @default.
• W3204244704 cites W2975712713 @default.
• W3204244704 cites W2985108934 @default.
• W3204244704 cites W2988070836 @default.
• W3204244704 cites W2994144272 @default.
• W3204244704 cites W3016395792 @default.
• W3204244704 cites W3047357290 @default.
• W3204244704 cites W3090287616 @default.
• W3204244704 cites W3101036738 @default.
• W3204244704 cites W3102606411 @default.
• W3204244704 cites W3102816259 @default.
• W3204244704 doi "https://doi.org/10.1145/3472456.3472508" @default.
• W3204244704 hasPublicationYear "2021" @default.
• W3204244704 type Work @default.
• W3204244704 sameAs 3204244704 @default.
• W3204244704 citedByCount "4" @default.
• W3204244704 countsByYear W32042447042022 @default.
• W3204244704 countsByYear W32042447042023 @default.
• W3204244704 crossrefType "proceedings-article" @default.
• W3204244704 hasAuthorship W3204244704A5000688713 @default.
• W3204244704 hasAuthorship W3204244704A5006729432 @default.
• W3204244704 hasAuthorship W3204244704A5007572801 @default.
• W3204244704 hasAuthorship W3204244704A5042353509 @default.
• W3204244704 hasAuthorship W3204244704A5080680105 @default.
• W3204244704 hasBestOaLocation W32042447042 @default.
• W3204244704 hasConcept C111919701 @default.
• W3204244704 hasConcept C11413529 @default.
• W3204244704 hasConcept C115961682 @default.
• W3204244704 hasConcept C13481523 @default.
• W3204244704 hasConcept C153258448 @default.
• W3204244704 hasConcept C154945302 @default.
• W3204244704 hasConcept C159985019 @default.
• W3204244704 hasConcept C162324750 @default.
• W3204244704 hasConcept C180016635 @default.
• W3204244704 hasConcept C192562407 @default.
• W3204244704 hasConcept C206688291 @default.
• W3204244704 hasConcept C26517878 @default.
• W3204244704 hasConcept C2777303404 @default.
• W3204244704 hasConcept C2779960059 @default.
• W3204244704 hasConcept C28855332 @default.
• W3204244704 hasConcept C38652104 @default.
• W3204244704 hasConcept C41008148 @default.
• W3204244704 hasConcept C45374587 @default.
• W3204244704 hasConcept C50522688 @default.
• W3204244704 hasConcept C50644808 @default.
• W3204244704 hasConcept C57869625 @default.
• W3204244704 hasConcept C78548338 @default.
• W3204244704 hasConcept C9417928 @default.
• W3204244704 hasConcept C94835093 @default.
• W3204244704 hasConceptScore W3204244704C111919701 @default.
• W3204244704 hasConceptScore W3204244704C11413529 @default.
• W3204244704 hasConceptScore W3204244704C115961682 @default.
• W3204244704 hasConceptScore W3204244704C13481523 @default.
• W3204244704 hasConceptScore W3204244704C153258448 @default.
• W3204244704 hasConceptScore W3204244704C154945302 @default.
• W3204244704 hasConceptScore W3204244704C159985019 @default.
• W3204244704 hasConceptScore W3204244704C162324750 @default.
• W3204244704 hasConceptScore W3204244704C180016635 @default.
• W3204244704 hasConceptScore W3204244704C192562407 @default.
• W3204244704 hasConceptScore W3204244704C206688291 @default.
• W3204244704 hasConceptScore W3204244704C26517878 @default.
• W3204244704 hasConceptScore W3204244704C2777303404 @default.
• W3204244704 hasConceptScore W3204244704C2779960059 @default.
• W3204244704 hasConceptScore W3204244704C28855332 @default.
• W3204244704 hasConceptScore W3204244704C38652104 @default.
• W3204244704 hasConceptScore W3204244704C41008148 @default.
• W3204244704 hasConceptScore W3204244704C45374587 @default.
• W3204244704 hasConceptScore W3204244704C50522688 @default.
• W3204244704 hasConceptScore W3204244704C50644808 @default.
• W3204244704 hasConceptScore W3204244704C57869625 @default.
• W3204244704 hasConceptScore W3204244704C78548338 @default.
• W3204244704 hasConceptScore W3204244704C9417928 @default.
• W3204244704 hasConceptScore W3204244704C94835093 @default.
• W3204244704 hasLocation W32042447041 @default.
• W3204244704 hasLocation W32042447042 @default.
• W3204244704 hasOpenAccess W3204244704 @default.
• W3204244704 hasPrimaryLocation W32042447041 @default.
• W3204244704 hasRelatedWork W2895097035 @default.
• W3204244704 hasRelatedWork W3000432941 @default.
• W3204244704 hasRelatedWork W3007528421 @default.
• W3204244704 hasRelatedWork W3037172934 @default.
• W3204244704 hasRelatedWork W3100294256 @default.
• W3204244704 hasRelatedWork W3173941299 @default.

• next