Matches in SemOpenAlex for { <https://semopenalex.org/work/W3114514717> ?p ?o ?g. }
Showing items 1 to 93 of
93
with 100 items per page.
- W3114514717 abstract "Decoupled learning is a branch of model parallelism which parallelizes the training of a network by splitting it depth-wise into multiple modules. Techniques from decoupled learning usually lead to stale gradient effect because of their asynchronous implementation, thereby causing performance degradation. In this paper, we propose an accumulated decoupled learning (ADL) which incorporates the gradient accumulation technique to mitigate the stale gradient effect. We give both theoretical and empirical evidences regarding how the gradient staleness can be reduced. We prove that the proposed method can converge to critical points, i.e., the gradients converge to 0, in spite of its asynchronous nature. Empirical validation is provided by training deep convolutional neural networks to perform classification tasks on CIFAR-10 and ImageNet datasets. The ADL is shown to outperform several state-of-the-arts in the classification tasks, and is the fastest among the compared methods." @default.
- W3114514717 created "2021-01-05" @default.
- W3114514717 creator A5018473729 @default.
- W3114514717 creator A5049506273 @default.
- W3114514717 creator A5061256037 @default.
- W3114514717 date "2020-12-03" @default.
- W3114514717 modified "2023-09-23" @default.
- W3114514717 title "Accumulated Decoupled Learning: Mitigating Gradient Staleness in Inter-Layer Model Parallelization." @default.
- W3114514717 cites W2112796928 @default.
- W3114514717 cites W2194775991 @default.
- W3114514717 cites W2401231614 @default.
- W3114514717 cites W2516591743 @default.
- W3114514717 cites W2523060838 @default.
- W3114514717 cites W2549139847 @default.
- W3114514717 cites W2619184049 @default.
- W3114514717 cites W2622263826 @default.
- W3114514717 cites W2626580042 @default.
- W3114514717 cites W2787998955 @default.
- W3114514717 cites W2812009592 @default.
- W3114514717 cites W2884700152 @default.
- W3114514717 cites W2907407288 @default.
- W3114514717 cites W2911586496 @default.
- W3114514717 cites W2912855321 @default.
- W3114514717 cites W2952388062 @default.
- W3114514717 cites W2952857865 @default.
- W3114514717 cites W2963373778 @default.
- W3114514717 cites W2963420686 @default.
- W3114514717 cites W2963446712 @default.
- W3114514717 cites W2970971581 @default.
- W3114514717 cites W2991040477 @default.
- W3114514717 cites W3035617116 @default.
- W3114514717 cites W3118608800 @default.
- W3114514717 cites W3121926921 @default.
- W3114514717 cites W778657980 @default.
- W3114514717 hasPublicationYear "2020" @default.
- W3114514717 type Work @default.
- W3114514717 sameAs 3114514717 @default.
- W3114514717 citedByCount "0" @default.
- W3114514717 crossrefType "posted-content" @default.
- W3114514717 hasAuthorship W3114514717A5018473729 @default.
- W3114514717 hasAuthorship W3114514717A5049506273 @default.
- W3114514717 hasAuthorship W3114514717A5061256037 @default.
- W3114514717 hasConcept C108583219 @default.
- W3114514717 hasConcept C151319957 @default.
- W3114514717 hasConcept C154945302 @default.
- W3114514717 hasConcept C173608175 @default.
- W3114514717 hasConcept C178790620 @default.
- W3114514717 hasConcept C185592680 @default.
- W3114514717 hasConcept C2779227376 @default.
- W3114514717 hasConcept C2781172179 @default.
- W3114514717 hasConcept C31258907 @default.
- W3114514717 hasConcept C41008148 @default.
- W3114514717 hasConcept C50644808 @default.
- W3114514717 hasConcept C81363708 @default.
- W3114514717 hasConceptScore W3114514717C108583219 @default.
- W3114514717 hasConceptScore W3114514717C151319957 @default.
- W3114514717 hasConceptScore W3114514717C154945302 @default.
- W3114514717 hasConceptScore W3114514717C173608175 @default.
- W3114514717 hasConceptScore W3114514717C178790620 @default.
- W3114514717 hasConceptScore W3114514717C185592680 @default.
- W3114514717 hasConceptScore W3114514717C2779227376 @default.
- W3114514717 hasConceptScore W3114514717C2781172179 @default.
- W3114514717 hasConceptScore W3114514717C31258907 @default.
- W3114514717 hasConceptScore W3114514717C41008148 @default.
- W3114514717 hasConceptScore W3114514717C50644808 @default.
- W3114514717 hasConceptScore W3114514717C81363708 @default.
- W3114514717 hasLocation W31145147171 @default.
- W3114514717 hasOpenAccess W3114514717 @default.
- W3114514717 hasPrimaryLocation W31145147171 @default.
- W3114514717 hasRelatedWork W2617019596 @default.
- W3114514717 hasRelatedWork W2617766261 @default.
- W3114514717 hasRelatedWork W2943847545 @default.
- W3114514717 hasRelatedWork W2952369090 @default.
- W3114514717 hasRelatedWork W2962710991 @default.
- W3114514717 hasRelatedWork W2971653326 @default.
- W3114514717 hasRelatedWork W2994911616 @default.
- W3114514717 hasRelatedWork W3013308737 @default.
- W3114514717 hasRelatedWork W3024699729 @default.
- W3114514717 hasRelatedWork W3035617116 @default.
- W3114514717 hasRelatedWork W3048804513 @default.
- W3114514717 hasRelatedWork W3087165870 @default.
- W3114514717 hasRelatedWork W3122102896 @default.
- W3114514717 hasRelatedWork W3130801519 @default.
- W3114514717 hasRelatedWork W3131685502 @default.
- W3114514717 hasRelatedWork W3138920078 @default.
- W3114514717 hasRelatedWork W3174871729 @default.
- W3114514717 hasRelatedWork W3198707348 @default.
- W3114514717 hasRelatedWork W3204507145 @default.
- W3114514717 hasRelatedWork W3205035978 @default.
- W3114514717 isParatext "false" @default.
- W3114514717 isRetracted "false" @default.
- W3114514717 magId "3114514717" @default.
- W3114514717 workType "article" @default.