FRINK Linked Data Fragments server

Matches in SemOpenAlex for { <https://semopenalex.org/work/W2973606585> ?p ?o ?g. }

• W2973606585 abstract "Distributed deep learning training usually adopts All-Reduce as the synchronization mechanism for data parallel algorithms due to its high performance in homogeneous environment. However, its performance is bounded by the slowest worker among all workers, and is significantly slower in heterogeneous situations. AD-PSGD, a newly proposed synchronization method which provides numerically fast convergence and heterogeneity tolerance, suffers from deadlock issues and high synchronization overhead. Is it possible to get the best of both worlds - designing a distributed training method that has both high performance as All-Reduce in homogeneous environment and good heterogeneity tolerance as AD-PSGD? In this paper, we propose Ripples, a high-performance heterogeneity-aware asynchronous decentralized training approach. We achieve the above goal with intensive synchronization optimization, emphasizing the interplay between algorithm and system implementation. To reduce synchronization cost, we propose a novel communication primitive Partial All-Reduce that allows a large group of workers to synchronize quickly. To reduce synchronization conflict, we propose static group scheduling in homogeneous environment and simple techniques (Group Buffer and Group Division) to avoid conflicts with slightly reduced randomness. Our experiments show that in homogeneous environment, Ripples is 1.1 times faster than the state-of-the-art implementation of All-Reduce, 5.1 times faster than Parameter Server and 4.3 times faster than AD-PSGD. In a heterogeneous setting, Ripples shows 2 times speedup over All-Reduce, and still obtains 3 times speedup over the Parameter Server baseline." @default.
• W2973606585 created "2019-09-26" @default.
• W2973606585 creator A5026252669 @default.
• W2973606585 creator A5047215143 @default.
• W2973606585 creator A5051409603 @default.
• W2973606585 creator A5066744881 @default.
• W2973606585 date "2019-09-17" @default.
• W2973606585 modified "2023-09-27" @default.
• W2973606585 title "Heterogeneity-Aware Asynchronous Decentralized Training" @default.
• W2973606585 cites W1575350781 @default.
• W2973606585 cites W1788418780 @default.
• W2973606585 cites W1825216778 @default.
• W2973606585 cites W1982063824 @default.
• W2973606585 cites W2018047324 @default.
• W2973606585 cites W2057332538 @default.
• W2973606585 cites W2060393849 @default.
• W2973606585 cites W2097117768 @default.
• W2973606585 cites W2117539524 @default.
• W2973606585 cites W2119391823 @default.
• W2973606585 cites W2132737349 @default.
• W2973606585 cites W2138243089 @default.
• W2973606585 cites W2162390675 @default.
• W2973606585 cites W2184045248 @default.
• W2973606585 cites W2186615578 @default.
• W2973606585 cites W2257979135 @default.
• W2973606585 cites W2302255633 @default.
• W2973606585 cites W2336650964 @default.
• W2973606585 cites W2402144811 @default.
• W2973606585 cites W2429966330 @default.
• W2973606585 cites W2604783387 @default.
• W2973606585 cites W2612026221 @default.
• W2973606585 cites W2622263826 @default.
• W2973606585 cites W2787998955 @default.
• W2973606585 cites W2789243870 @default.
• W2973606585 cites W2794670651 @default.
• W2973606585 cites W2807147113 @default.
• W2973606585 cites W2884700152 @default.
• W2973606585 cites W2884711234 @default.
• W2973606585 cites W2889676205 @default.
• W2973606585 cites W2894152871 @default.
• W2973606585 cites W2895512264 @default.
• W2973606585 cites W2895763047 @default.
• W2973606585 cites W2911546574 @default.
• W2973606585 cites W2915943718 @default.
• W2973606585 cites W2926655273 @default.
• W2973606585 cites W2951781666 @default.
• W2973606585 cites W2952046647 @default.
• W2973606585 cites W2962700998 @default.
• W2973606585 cites W2962835968 @default.
• W2973606585 cites W2962863496 @default.
• W2973606585 cites W2963228337 @default.
• W2973606585 cites W2963717807 @default.
• W2973606585 cites W2972087877 @default.
• W2973606585 cites W3037047862 @default.
• W2973606585 cites W3098222317 @default.
• W2973606585 cites W3118608800 @default.
• W2973606585 cites W2517695692 @default.
• W2973606585 hasPublicationYear "2019" @default.
• W2973606585 type Work @default.
• W2973606585 sameAs 2973606585 @default.
• W2973606585 citedByCount "6" @default.
• W2973606585 countsByYear W29736065852019 @default.
• W2973606585 countsByYear W29736065852020 @default.
• W2973606585 countsByYear W29736065852021 @default.
• W2973606585 crossrefType "posted-content" @default.
• W2973606585 hasAuthorship W2973606585A5026252669 @default.
• W2973606585 hasAuthorship W2973606585A5047215143 @default.
• W2973606585 hasAuthorship W2973606585A5051409603 @default.
• W2973606585 hasAuthorship W2973606585A5066744881 @default.
• W2973606585 hasConcept C108734733 @default.
• W2973606585 hasConcept C111919701 @default.
• W2973606585 hasConcept C114614502 @default.
• W2973606585 hasConcept C120314980 @default.
• W2973606585 hasConcept C126255220 @default.
• W2973606585 hasConcept C127162648 @default.
• W2973606585 hasConcept C13280743 @default.
• W2973606585 hasConcept C151319957 @default.
• W2973606585 hasConcept C173608175 @default.
• W2973606585 hasConcept C185798385 @default.
• W2973606585 hasConcept C205649164 @default.
• W2973606585 hasConcept C206729178 @default.
• W2973606585 hasConcept C2778562939 @default.
• W2973606585 hasConcept C2779960059 @default.
• W2973606585 hasConcept C31258907 @default.
• W2973606585 hasConcept C33923547 @default.
• W2973606585 hasConcept C41008148 @default.
• W2973606585 hasConcept C66882249 @default.
• W2973606585 hasConcept C68339613 @default.
• W2973606585 hasConcept C98045186 @default.
• W2973606585 hasConceptScore W2973606585C108734733 @default.
• W2973606585 hasConceptScore W2973606585C111919701 @default.
• W2973606585 hasConceptScore W2973606585C114614502 @default.
• W2973606585 hasConceptScore W2973606585C120314980 @default.
• W2973606585 hasConceptScore W2973606585C126255220 @default.
• W2973606585 hasConceptScore W2973606585C127162648 @default.
• W2973606585 hasConceptScore W2973606585C13280743 @default.
• W2973606585 hasConceptScore W2973606585C151319957 @default.
• W2973606585 hasConceptScore W2973606585C173608175 @default.
• W2973606585 hasConceptScore W2973606585C185798385 @default.
• W2973606585 hasConceptScore W2973606585C205649164 @default.

• next