FRINK Linked Data Fragments server

Matches in SemOpenAlex for { <https://semopenalex.org/work/W3040963243> ?p ?o ?g. }

• W3040963243 abstract "Asynchronous distributed algorithms are a popular way to reduce synchronization costs in large-scale optimization, and in particular for neural network training. However, for nonsmooth and nonconvex objectives, few convergence guarantees exist beyond cases where closed-form proximal operator solutions are available. As most popular contemporary deep neural networks lead to nonsmooth and nonconvex objectives, there is now a pressing need for such convergence guarantees. In this paper, we analyze for the first time the convergence of stochastic asynchronous optimization for this general class of objectives. In particular, we focus on stochastic subgradient methods allowing for block variable partitioning, where the shared-memory-based model is asynchronously updated by concurrent processes. To this end, we first introduce a probabilistic model which captures key features of real asynchronous scheduling between concurrent processes; under this model, we establish convergence with probability one to an invariant set for stochastic subgradient methods with momentum. From the practical perspective, one issue with the family of methods we consider is that it is not efficiently supported by machine learning frameworks, as they mostly focus on distributed data-parallel strategies. To address this, we propose a new implementation strategy for shared-memory based training of deep neural networks, whereby concurrent parameter servers are utilized to train a partitioned but shared model in single- and multi-GPU settings. Based on this implementation, we achieve on average 1.2x speed-up in comparison to state-of-the-art training methods for popular image classification tasks without compromising accuracy." @default.
• W3040963243 created "2020-07-16" @default.
• W3040963243 creator A5038057850 @default.
• W3040963243 creator A5043227337 @default.
• W3040963243 creator A5066151990 @default.
• W3040963243 creator A5083822059 @default.
• W3040963243 date "2019-05-28" @default.
• W3040963243 modified "2023-10-16" @default.
• W3040963243 title "Asynchronous Optimization Methods for Efficient Training of Deep Neural Networks with Guarantees" @default.
• W3040963243 cites W104184427 @default.
• W3040963243 cites W1499021337 @default.
• W3040963243 cites W1986324933 @default.
• W3040963243 cites W1994616650 @default.
• W3040963243 cites W2042132284 @default.
• W3040963243 cites W2079482358 @default.
• W3040963243 cites W2083842231 @default.
• W3040963243 cites W2091787562 @default.
• W3040963243 cites W2126794261 @default.
• W3040963243 cites W2132565565 @default.
• W3040963243 cites W2138243089 @default.
• W3040963243 cites W2146502635 @default.
• W3040963243 cites W2194775991 @default.
• W3040963243 cites W2335728318 @default.
• W3040963243 cites W2495560524 @default.
• W3040963243 cites W2622263826 @default.
• W3040963243 cites W2787899480 @default.
• W3040963243 cites W2798818710 @default.
• W3040963243 cites W2800922801 @default.
• W3040963243 cites W2807147113 @default.
• W3040963243 cites W2899771611 @default.
• W3040963243 cites W2962760808 @default.
• W3040963243 cites W2963125010 @default.
• W3040963243 cites W2963433607 @default.
• W3040963243 cites W2963446712 @default.
• W3040963243 cites W2963614065 @default.
• W3040963243 cites W2963792515 @default.
• W3040963243 cites W2963903325 @default.
• W3040963243 cites W2982000031 @default.
• W3040963243 cites W3118608800 @default.
• W3040963243 cites W594357522 @default.
• W3040963243 cites W3037282472 @default.
• W3040963243 doi "https://doi.org/10.48550/arxiv.1905.11845" @default.
• W3040963243 hasPublicationYear "2019" @default.
• W3040963243 type Work @default.
• W3040963243 sameAs 3040963243 @default.
• W3040963243 citedByCount "1" @default.
• W3040963243 countsByYear W30409632432020 @default.
• W3040963243 crossrefType "posted-content" @default.
• W3040963243 hasAuthorship W3040963243A5038057850 @default.
• W3040963243 hasAuthorship W3040963243A5043227337 @default.
• W3040963243 hasAuthorship W3040963243A5066151990 @default.
• W3040963243 hasAuthorship W3040963243A5083822059 @default.
• W3040963243 hasBestOaLocation W30409632431 @default.
• W3040963243 hasConcept C119857082 @default.
• W3040963243 hasConcept C120314980 @default.
• W3040963243 hasConcept C126255220 @default.
• W3040963243 hasConcept C151319957 @default.
• W3040963243 hasConcept C154945302 @default.
• W3040963243 hasConcept C158968445 @default.
• W3040963243 hasConcept C162324750 @default.
• W3040963243 hasConcept C2777303404 @default.
• W3040963243 hasConcept C31258907 @default.
• W3040963243 hasConcept C33923547 @default.
• W3040963243 hasConcept C41008148 @default.
• W3040963243 hasConcept C49937458 @default.
• W3040963243 hasConcept C50522688 @default.
• W3040963243 hasConcept C50644808 @default.
• W3040963243 hasConceptScore W3040963243C119857082 @default.
• W3040963243 hasConceptScore W3040963243C120314980 @default.
• W3040963243 hasConceptScore W3040963243C126255220 @default.
• W3040963243 hasConceptScore W3040963243C151319957 @default.
• W3040963243 hasConceptScore W3040963243C154945302 @default.
• W3040963243 hasConceptScore W3040963243C158968445 @default.
• W3040963243 hasConceptScore W3040963243C162324750 @default.
• W3040963243 hasConceptScore W3040963243C2777303404 @default.
• W3040963243 hasConceptScore W3040963243C31258907 @default.
• W3040963243 hasConceptScore W3040963243C33923547 @default.
• W3040963243 hasConceptScore W3040963243C41008148 @default.
• W3040963243 hasConceptScore W3040963243C49937458 @default.
• W3040963243 hasConceptScore W3040963243C50522688 @default.
• W3040963243 hasConceptScore W3040963243C50644808 @default.
• W3040963243 hasLocation W30409632431 @default.
• W3040963243 hasLocation W30409632432 @default.
• W3040963243 hasLocation W30409632433 @default.
• W3040963243 hasLocation W30409632434 @default.
• W3040963243 hasOpenAccess W3040963243 @default.
• W3040963243 hasPrimaryLocation W30409632431 @default.
• W3040963243 hasRelatedWork W1971290991 @default.
• W3040963243 hasRelatedWork W2000743687 @default.
• W3040963243 hasRelatedWork W2037613239 @default.
• W3040963243 hasRelatedWork W2059050877 @default.
• W3040963243 hasRelatedWork W2359120930 @default.
• W3040963243 hasRelatedWork W2390469630 @default.
• W3040963243 hasRelatedWork W2781777036 @default.
• W3040963243 hasRelatedWork W4236372925 @default.
• W3040963243 hasRelatedWork W4361865857 @default.
• W3040963243 hasRelatedWork W4378770495 @default.
• W3040963243 isParatext "false" @default.
• W3040963243 isRetracted "false" @default.
• W3040963243 magId "3040963243" @default.

• next