Matches in SemOpenAlex for { <https://semopenalex.org/work/W4302028822> ?p ?o ?g. }
Showing items 1 to 72 of
72
with 100 items per page.
- W4302028822 abstract "This work addresses the instability in asynchronous data parallel optimization. It does so by introducing a novel distributed optimizer which is able to efficiently optimize a centralized model under communication constraints. The optimizer achieves this by pushing a normalized sequence of first-order gradients to a parameter server. This implies that the magnitude of a worker delta is smaller compared to an accumulated gradient, and provides a better direction towards a minimum compared to first-order gradients, which in turn also forces possible implicit momentum fluctuations to be more aligned since we make the assumption that all workers contribute towards a single minima. As a result, our approach mitigates the parameter staleness problem more effectively since staleness in asynchrony induces (implicit) momentum, and achieves a better convergence rate compared to other optimizers such as asynchronous EASGD and DynSGD, which we show empirically." @default.
- W4302028822 created "2022-10-06" @default.
- W4302028822 creator A5010354377 @default.
- W4302028822 creator A5053292274 @default.
- W4302028822 creator A5088910768 @default.
- W4302028822 date "2017-10-06" @default.
- W4302028822 modified "2023-10-16" @default.
- W4302028822 title "Accumulated Gradient Normalization" @default.
- W4302028822 doi "https://doi.org/10.48550/arxiv.1710.02368" @default.
- W4302028822 hasPublicationYear "2017" @default.
- W4302028822 type Work @default.
- W4302028822 citedByCount "0" @default.
- W4302028822 crossrefType "posted-content" @default.
- W4302028822 hasAuthorship W4302028822A5010354377 @default.
- W4302028822 hasAuthorship W4302028822A5053292274 @default.
- W4302028822 hasAuthorship W4302028822A5088910768 @default.
- W4302028822 hasBestOaLocation W43020288221 @default.
- W4302028822 hasConcept C10138342 @default.
- W4302028822 hasConcept C11413529 @default.
- W4302028822 hasConcept C126255220 @default.
- W4302028822 hasConcept C127162648 @default.
- W4302028822 hasConcept C134306372 @default.
- W4302028822 hasConcept C136886441 @default.
- W4302028822 hasConcept C144024400 @default.
- W4302028822 hasConcept C151319957 @default.
- W4302028822 hasConcept C162324750 @default.
- W4302028822 hasConcept C186633575 @default.
- W4302028822 hasConcept C19165224 @default.
- W4302028822 hasConcept C2777303404 @default.
- W4302028822 hasConcept C2779019669 @default.
- W4302028822 hasConcept C31258907 @default.
- W4302028822 hasConcept C33923547 @default.
- W4302028822 hasConcept C41008148 @default.
- W4302028822 hasConcept C50522688 @default.
- W4302028822 hasConcept C57869625 @default.
- W4302028822 hasConcept C60718061 @default.
- W4302028822 hasConceptScore W4302028822C10138342 @default.
- W4302028822 hasConceptScore W4302028822C11413529 @default.
- W4302028822 hasConceptScore W4302028822C126255220 @default.
- W4302028822 hasConceptScore W4302028822C127162648 @default.
- W4302028822 hasConceptScore W4302028822C134306372 @default.
- W4302028822 hasConceptScore W4302028822C136886441 @default.
- W4302028822 hasConceptScore W4302028822C144024400 @default.
- W4302028822 hasConceptScore W4302028822C151319957 @default.
- W4302028822 hasConceptScore W4302028822C162324750 @default.
- W4302028822 hasConceptScore W4302028822C186633575 @default.
- W4302028822 hasConceptScore W4302028822C19165224 @default.
- W4302028822 hasConceptScore W4302028822C2777303404 @default.
- W4302028822 hasConceptScore W4302028822C2779019669 @default.
- W4302028822 hasConceptScore W4302028822C31258907 @default.
- W4302028822 hasConceptScore W4302028822C33923547 @default.
- W4302028822 hasConceptScore W4302028822C41008148 @default.
- W4302028822 hasConceptScore W4302028822C50522688 @default.
- W4302028822 hasConceptScore W4302028822C57869625 @default.
- W4302028822 hasConceptScore W4302028822C60718061 @default.
- W4302028822 hasLocation W43020288221 @default.
- W4302028822 hasLocation W43020288222 @default.
- W4302028822 hasOpenAccess W4302028822 @default.
- W4302028822 hasPrimaryLocation W43020288221 @default.
- W4302028822 hasRelatedWork W2156376770 @default.
- W4302028822 hasRelatedWork W2763726490 @default.
- W4302028822 hasRelatedWork W2806788360 @default.
- W4302028822 hasRelatedWork W2963080841 @default.
- W4302028822 hasRelatedWork W2963251229 @default.
- W4302028822 hasRelatedWork W2964667463 @default.
- W4302028822 hasRelatedWork W3214637427 @default.
- W4302028822 hasRelatedWork W4221157705 @default.
- W4302028822 hasRelatedWork W4302028822 @default.
- W4302028822 hasRelatedWork W4362511394 @default.
- W4302028822 isParatext "false" @default.
- W4302028822 isRetracted "false" @default.
- W4302028822 workType "article" @default.