Matches in SemOpenAlex for { <https://semopenalex.org/work/W4366393886> ?p ?o ?g. }
Showing items 1 to 83 of
83
with 100 items per page.
- W4366393886 endingPage "109777" @default.
- W4366393886 startingPage "109777" @default.
- W4366393886 abstract "Efficient communication is crucial to Distributed Machine Learning (DML). In this work, we propose an approach jointing Data Formatting and Sparsification (DFS) to optimize the communication in DML systems based on the parameter server framework. By doing so, we can reduce the time to transmit (aggregated) gradients between the parameter server and workers, and consequently the time to complete training jobs. More specifically, in DFS, every worker first tries to derive as many blocks with all-zero gradients as possible via sparsification, and transmits gradients block by block in a streaming fashion. By skipping blocks with all-zero gradients, we can reduce the communication cost for gradient transmission. Different from previous works on optimizing the communication in DML systems, DFS has three distinct features: (i). it dynamically determines the gradient block size; (ii). it takes into consideration both the data transfer from workers to the parameter server and that from the parameter server to workers; and (iii). it jointly optimizes the data formatting and sparsification. In other words, it performs sparsification in the way that helps form more (or larger) all-zero blocks and save more communication cost. By implementing DFS on a real testbed, we find that it can reduce the time to train a ResNet-18 model by 74.12%. Through extensive simulations, we demonstrate that DFS outperforms the state-of-the-art technique, i.e., OmniReduce (Fei et al., 2021), by up to 87.17% in terms of reducing communication cost in DML systems." @default.
- W4366393886 created "2023-04-21" @default.
- W4366393886 creator A5026096129 @default.
- W4366393886 creator A5063184427 @default.
- W4366393886 creator A5074187380 @default.
- W4366393886 creator A5075889174 @default.
- W4366393886 date "2023-06-01" @default.
- W4366393886 modified "2023-09-23" @default.
- W4366393886 title "DFS: Joint data formatting and sparsification for efficient communication in Distributed Machine Learning" @default.
- W4366393886 cites W1542192632 @default.
- W4366393886 cites W2057332538 @default.
- W4366393886 cites W2060393849 @default.
- W4366393886 cites W2117130368 @default.
- W4366393886 cites W2130531694 @default.
- W4366393886 cites W2155893237 @default.
- W4366393886 cites W2160815625 @default.
- W4366393886 cites W2194775991 @default.
- W4366393886 cites W2268702383 @default.
- W4366393886 cites W2472333518 @default.
- W4366393886 cites W2498764059 @default.
- W4366393886 cites W2809353470 @default.
- W4366393886 cites W2906007643 @default.
- W4366393886 cites W2962739339 @default.
- W4366393886 cites W2985108934 @default.
- W4366393886 cites W3091097978 @default.
- W4366393886 cites W4235732943 @default.
- W4366393886 cites W4237515752 @default.
- W4366393886 doi "https://doi.org/10.1016/j.comnet.2023.109777" @default.
- W4366393886 hasPublicationYear "2023" @default.
- W4366393886 type Work @default.
- W4366393886 citedByCount "0" @default.
- W4366393886 crossrefType "journal-article" @default.
- W4366393886 hasAuthorship W4366393886A5026096129 @default.
- W4366393886 hasAuthorship W4366393886A5063184427 @default.
- W4366393886 hasAuthorship W4366393886A5074187380 @default.
- W4366393886 hasAuthorship W4366393886A5075889174 @default.
- W4366393886 hasConcept C111919701 @default.
- W4366393886 hasConcept C11413529 @default.
- W4366393886 hasConcept C120314980 @default.
- W4366393886 hasConcept C152043487 @default.
- W4366393886 hasConcept C2524010 @default.
- W4366393886 hasConcept C2776175482 @default.
- W4366393886 hasConcept C2777210771 @default.
- W4366393886 hasConcept C31258907 @default.
- W4366393886 hasConcept C31395832 @default.
- W4366393886 hasConcept C33923547 @default.
- W4366393886 hasConcept C41008148 @default.
- W4366393886 hasConcept C557945733 @default.
- W4366393886 hasConcept C79403827 @default.
- W4366393886 hasConcept C88006597 @default.
- W4366393886 hasConceptScore W4366393886C111919701 @default.
- W4366393886 hasConceptScore W4366393886C11413529 @default.
- W4366393886 hasConceptScore W4366393886C120314980 @default.
- W4366393886 hasConceptScore W4366393886C152043487 @default.
- W4366393886 hasConceptScore W4366393886C2524010 @default.
- W4366393886 hasConceptScore W4366393886C2776175482 @default.
- W4366393886 hasConceptScore W4366393886C2777210771 @default.
- W4366393886 hasConceptScore W4366393886C31258907 @default.
- W4366393886 hasConceptScore W4366393886C31395832 @default.
- W4366393886 hasConceptScore W4366393886C33923547 @default.
- W4366393886 hasConceptScore W4366393886C41008148 @default.
- W4366393886 hasConceptScore W4366393886C557945733 @default.
- W4366393886 hasConceptScore W4366393886C79403827 @default.
- W4366393886 hasConceptScore W4366393886C88006597 @default.
- W4366393886 hasLocation W43663938861 @default.
- W4366393886 hasOpenAccess W4366393886 @default.
- W4366393886 hasPrimaryLocation W43663938861 @default.
- W4366393886 hasRelatedWork W2096085985 @default.
- W4366393886 hasRelatedWork W2099096883 @default.
- W4366393886 hasRelatedWork W2104625236 @default.
- W4366393886 hasRelatedWork W2363683157 @default.
- W4366393886 hasRelatedWork W2364074001 @default.
- W4366393886 hasRelatedWork W2383503060 @default.
- W4366393886 hasRelatedWork W2387284662 @default.
- W4366393886 hasRelatedWork W2768101539 @default.
- W4366393886 hasRelatedWork W3026711068 @default.
- W4366393886 hasRelatedWork W4224231277 @default.
- W4366393886 hasVolume "229" @default.
- W4366393886 isParatext "false" @default.
- W4366393886 isRetracted "false" @default.
- W4366393886 workType "article" @default.