Matches in SemOpenAlex for { <https://semopenalex.org/work/W4290996544> ?p ?o ?g. }
Showing items 1 to 79 of
79
with 100 items per page.
- W4290996544 abstract "Distributed learning is widely used to accelerate the training of deep learning models, but it is known that communication efficiency limits the scalability of distributed learning systems. Current gradient compression techniques provide promising methods to reduce communication time, but the extra time incurred by compression is not negligible. After compression techniques are applied, the communication time is significantly reduced because the data size needed to communicate becomes much smaller, but compressing gradients is time-consuming and it becomes a new bottleneck. In this paper, we design and implement PipeCompress, a system to decouple compression and backpropagation operations into two processes and pipeline the two processes to hide compression time. We also propose a specialized inter-process communication mechanism based on the characteristics of DNN distributed training to improve the efficiency of passing messages between the two processes, which makes sure that the decoupling does not bring much extra inter-process communication time cost. As far as we know, this is the first work that notices the overhead of compression and pipelines backpropagation and compression operations to hide compression time in distributed learning. Experiments show that PipeCompress can significantly hide compression time, reduce iteration time, and accelerate the training process on various DNN models." @default.
- W4290996544 created "2022-08-13" @default.
- W4290996544 creator A5015855316 @default.
- W4290996544 creator A5030007975 @default.
- W4290996544 creator A5041059888 @default.
- W4290996544 creator A5075088248 @default.
- W4290996544 date "2022-05-16" @default.
- W4290996544 modified "2023-09-24" @default.
- W4290996544 title "PipeCompress: Accelerating Pipelined Communication for Distributed Deep Learning" @default.
- W4290996544 cites W2097117768 @default.
- W4290996544 cites W2194775991 @default.
- W4290996544 cites W2963786636 @default.
- W4290996544 cites W2975712713 @default.
- W4290996544 cites W3004495293 @default.
- W4290996544 cites W3047357290 @default.
- W4290996544 cites W3047537431 @default.
- W4290996544 doi "https://doi.org/10.1109/icc45855.2022.9839126" @default.
- W4290996544 hasPublicationYear "2022" @default.
- W4290996544 type Work @default.
- W4290996544 citedByCount "0" @default.
- W4290996544 crossrefType "proceedings-article" @default.
- W4290996544 hasAuthorship W4290996544A5015855316 @default.
- W4290996544 hasAuthorship W4290996544A5030007975 @default.
- W4290996544 hasAuthorship W4290996544A5041059888 @default.
- W4290996544 hasAuthorship W4290996544A5075088248 @default.
- W4290996544 hasConcept C101765175 @default.
- W4290996544 hasConcept C108583219 @default.
- W4290996544 hasConcept C111919701 @default.
- W4290996544 hasConcept C120314980 @default.
- W4290996544 hasConcept C149635348 @default.
- W4290996544 hasConcept C154945302 @default.
- W4290996544 hasConcept C155032097 @default.
- W4290996544 hasConcept C199360897 @default.
- W4290996544 hasConcept C2780513914 @default.
- W4290996544 hasConcept C31258907 @default.
- W4290996544 hasConcept C41008148 @default.
- W4290996544 hasConcept C43521106 @default.
- W4290996544 hasConcept C48044578 @default.
- W4290996544 hasConcept C50644808 @default.
- W4290996544 hasConcept C77088390 @default.
- W4290996544 hasConcept C78548338 @default.
- W4290996544 hasConcept C79403827 @default.
- W4290996544 hasConcept C98045186 @default.
- W4290996544 hasConceptScore W4290996544C101765175 @default.
- W4290996544 hasConceptScore W4290996544C108583219 @default.
- W4290996544 hasConceptScore W4290996544C111919701 @default.
- W4290996544 hasConceptScore W4290996544C120314980 @default.
- W4290996544 hasConceptScore W4290996544C149635348 @default.
- W4290996544 hasConceptScore W4290996544C154945302 @default.
- W4290996544 hasConceptScore W4290996544C155032097 @default.
- W4290996544 hasConceptScore W4290996544C199360897 @default.
- W4290996544 hasConceptScore W4290996544C2780513914 @default.
- W4290996544 hasConceptScore W4290996544C31258907 @default.
- W4290996544 hasConceptScore W4290996544C41008148 @default.
- W4290996544 hasConceptScore W4290996544C43521106 @default.
- W4290996544 hasConceptScore W4290996544C48044578 @default.
- W4290996544 hasConceptScore W4290996544C50644808 @default.
- W4290996544 hasConceptScore W4290996544C77088390 @default.
- W4290996544 hasConceptScore W4290996544C78548338 @default.
- W4290996544 hasConceptScore W4290996544C79403827 @default.
- W4290996544 hasConceptScore W4290996544C98045186 @default.
- W4290996544 hasFunder F4320321001 @default.
- W4290996544 hasFunder F4320337504 @default.
- W4290996544 hasLocation W42909965441 @default.
- W4290996544 hasOpenAccess W4290996544 @default.
- W4290996544 hasPrimaryLocation W42909965441 @default.
- W4290996544 hasRelatedWork W1596201972 @default.
- W4290996544 hasRelatedWork W1788737569 @default.
- W4290996544 hasRelatedWork W1967954938 @default.
- W4290996544 hasRelatedWork W2087937280 @default.
- W4290996544 hasRelatedWork W2124870959 @default.
- W4290996544 hasRelatedWork W2361361118 @default.
- W4290996544 hasRelatedWork W2364921833 @default.
- W4290996544 hasRelatedWork W2385146268 @default.
- W4290996544 hasRelatedWork W2394264333 @default.
- W4290996544 hasRelatedWork W94000989 @default.
- W4290996544 isParatext "false" @default.
- W4290996544 isRetracted "false" @default.
- W4290996544 workType "article" @default.