FRINK Linked Data Fragments server

Matches in SemOpenAlex for { <https://semopenalex.org/work/W2347281465> ?p ?o ?g. }

• W2347281465 abstract "We study the problem of how to distribute the training of large-scale deep learning models in the parallel computing environment. We propose a new distributed stochastic optimization method called Elastic Averaging SGD (EASGD). We analyze the convergence rate of the EASGD method in the synchronous scenario and compare its stability condition with the existing ADMM method in the round-robin scheme. An asynchronous and momentum variant of the EASGD method is applied to train deep convolutional neural networks for image classification on the CIFAR and ImageNet datasets. Our approach accelerates the training and furthermore achieves better test accuracy. It also requires a much smaller amount of communication than other common baseline approaches such as the DOWNPOUR method. We then investigate the limit in speedup of the initial and the asymptotic phase of the mini-batch SGD, the momentum SGD, and the EASGD methods. We find that the spread of the input data distribution has a big impact on their initial convergence rate and stability region. We also find a surprising connection between the momentum SGD and the EASGD method with a negative moving average rate. A non-convex case is also studied to understand when EASGD can get trapped by a saddle point. Finally, we scale up the EASGD method by using a tree structured network topology. We show empirically its advantage and challenge. We also establish a connection between the EASGD and the DOWNPOUR method with the classical Jacobi and the Gauss-Seidel method, thus unifying a class of distributed stochastic optimization methods." @default.
• W2347281465 created "2016-06-24" @default.
• W2347281465 creator A5071191599 @default.
• W2347281465 date "2016-05-07" @default.
• W2347281465 modified "2023-10-16" @default.
• W2347281465 title "Distributed stochastic optimization for deep learning (thesis)" @default.
• W2347281465 cites W104184427 @default.
• W2347281465 cites W130696423 @default.
• W2347281465 cites W1506295250 @default.
• W2347281465 cites W1506342804 @default.
• W2347281465 cites W1528719522 @default.
• W2347281465 cites W1560700853 @default.
• W2347281465 cites W1568229137 @default.
• W2347281465 cites W1568288633 @default.
• W2347281465 cites W1811750039 @default.
• W2347281465 cites W1899249567 @default.
• W2347281465 cites W1970997001 @default.
• W2347281465 cites W2009537245 @default.
• W2347281465 cites W2024484010 @default.
• W2347281465 cites W2072566913 @default.
• W2347281465 cites W2080631849 @default.
• W2347281465 cites W2086161653 @default.
• W2347281465 cites W2095705004 @default.
• W2347281465 cites W2106221286 @default.
• W2347281465 cites W2109339818 @default.
• W2347281465 cites W2117499659 @default.
• W2347281465 cites W2124768887 @default.
• W2347281465 cites W2130062883 @default.
• W2347281465 cites W2137731592 @default.
• W2347281465 cites W2140713321 @default.
• W2347281465 cites W2156118598 @default.
• W2347281465 cites W2164278908 @default.
• W2347281465 cites W2166706236 @default.
• W2347281465 cites W2167732364 @default.
• W2347281465 cites W2168231600 @default.
• W2347281465 cites W2173398862 @default.
• W2347281465 cites W2198403777 @default.
• W2347281465 cites W2257979135 @default.
• W2347281465 cites W2259324379 @default.
• W2347281465 cites W2284143396 @default.
• W2347281465 cites W2287011250 @default.
• W2347281465 cites W2289750073 @default.
• W2347281465 cites W2407022425 @default.
• W2347281465 cites W2414455940 @default.
• W2347281465 cites W2510516734 @default.
• W2347281465 cites W2949198759 @default.
• W2347281465 cites W2951488730 @default.
• W2347281465 cites W2951781666 @default.
• W2347281465 cites W2952033860 @default.
• W2347281465 cites W2952388062 @default.
• W2347281465 cites W2963225922 @default.
• W2347281465 cites W2963542991 @default.
• W2347281465 cites W2964181194 @default.
• W2347281465 cites W4919037 @default.
• W2347281465 doi "https://doi.org/10.48550/arxiv.1605.02216" @default.
• W2347281465 hasPublicationYear "2016" @default.
• W2347281465 type Work @default.
• W2347281465 sameAs 2347281465 @default.
• W2347281465 citedByCount "1" @default.
• W2347281465 countsByYear W23472814652022 @default.
• W2347281465 crossrefType "posted-content" @default.
• W2347281465 hasAuthorship W2347281465A5071191599 @default.
• W2347281465 hasBestOaLocation W23472814651 @default.
• W2347281465 hasConcept C108583219 @default.
• W2347281465 hasConcept C112972136 @default.
• W2347281465 hasConcept C11413529 @default.
• W2347281465 hasConcept C119857082 @default.
• W2347281465 hasConcept C126255220 @default.
• W2347281465 hasConcept C127162648 @default.
• W2347281465 hasConcept C151319957 @default.
• W2347281465 hasConcept C154945302 @default.
• W2347281465 hasConcept C162324750 @default.
• W2347281465 hasConcept C173608175 @default.
• W2347281465 hasConcept C2777303404 @default.
• W2347281465 hasConcept C31258907 @default.
• W2347281465 hasConcept C33923547 @default.
• W2347281465 hasConcept C41008148 @default.
• W2347281465 hasConcept C50522688 @default.
• W2347281465 hasConcept C57869625 @default.
• W2347281465 hasConcept C68339613 @default.
• W2347281465 hasConcept C81363708 @default.
• W2347281465 hasConceptScore W2347281465C108583219 @default.
• W2347281465 hasConceptScore W2347281465C112972136 @default.
• W2347281465 hasConceptScore W2347281465C11413529 @default.
• W2347281465 hasConceptScore W2347281465C119857082 @default.
• W2347281465 hasConceptScore W2347281465C126255220 @default.
• W2347281465 hasConceptScore W2347281465C127162648 @default.
• W2347281465 hasConceptScore W2347281465C151319957 @default.
• W2347281465 hasConceptScore W2347281465C154945302 @default.
• W2347281465 hasConceptScore W2347281465C162324750 @default.
• W2347281465 hasConceptScore W2347281465C173608175 @default.
• W2347281465 hasConceptScore W2347281465C2777303404 @default.
• W2347281465 hasConceptScore W2347281465C31258907 @default.
• W2347281465 hasConceptScore W2347281465C33923547 @default.
• W2347281465 hasConceptScore W2347281465C41008148 @default.
• W2347281465 hasConceptScore W2347281465C50522688 @default.
• W2347281465 hasConceptScore W2347281465C57869625 @default.
• W2347281465 hasConceptScore W2347281465C68339613 @default.
• W2347281465 hasConceptScore W2347281465C81363708 @default.
• W2347281465 hasLocation W23472814651 @default.

• next