Matches in SemOpenAlex for { <https://semopenalex.org/work/W2987364544> ?p ?o ?g. }
- W2987364544 abstract "Although ADAM is a very popular algorithm for optimizing the weights of neural networks, it has been recently shown that it can diverge even in simple convex optimization examples. Several variants of ADAM have been proposed to circumvent this convergence issue. In this work, we study the ADAM algorithm for smooth nonconvex optimization under a boundedness assumption on the adaptive learning rate. The bound on the adaptive step size depends on the Lipschitz constant of the gradient of the objective function and provides safe theoretical adaptive step sizes. Under this boundedness assumption, we show a novel first order convergence rate result in both deterministic and stochastic contexts. Furthermore, we establish convergence rates of the function value sequence using the Kurdyka-Łojasiewicz property." @default.
- W2987364544 created "2019-11-22" @default.
- W2987364544 creator A5024802752 @default.
- W2987364544 creator A5065250840 @default.
- W2987364544 date "2019-11-15" @default.
- W2987364544 modified "2023-10-06" @default.
- W2987364544 title "Convergence Analysis of a Momentum Algorithm with Adaptive Step Size for Non Convex Optimization" @default.
- W2987364544 cites W104184427 @default.
- W2987364544 cites W1518461880 @default.
- W2987364544 cites W1815076433 @default.
- W2987364544 cites W1967138577 @default.
- W2987364544 cites W1988720110 @default.
- W2987364544 cites W1994616650 @default.
- W2987364544 cites W2000462146 @default.
- W2987364544 cites W2004160833 @default.
- W2987364544 cites W2022541060 @default.
- W2987364544 cites W2027982384 @default.
- W2987364544 cites W2068357778 @default.
- W2987364544 cites W2077612096 @default.
- W2987364544 cites W2078853581 @default.
- W2987364544 cites W2129732816 @default.
- W2987364544 cites W2146502635 @default.
- W2987364544 cites W2547280488 @default.
- W2987364544 cites W2627272094 @default.
- W2987364544 cites W2785523195 @default.
- W2987364544 cites W2789621689 @default.
- W2987364544 cites W2807697622 @default.
- W2987364544 cites W2808042107 @default.
- W2987364544 cites W2886463271 @default.
- W2987364544 cites W2887729414 @default.
- W2987364544 cites W2891952073 @default.
- W2987364544 cites W2894692220 @default.
- W2987364544 cites W2903382683 @default.
- W2987364544 cites W2907225497 @default.
- W2987364544 cites W2912323147 @default.
- W2987364544 cites W2916621068 @default.
- W2987364544 cites W2945586457 @default.
- W2987364544 cites W2946511237 @default.
- W2987364544 cites W2947452777 @default.
- W2987364544 cites W2947930523 @default.
- W2987364544 cites W2962734306 @default.
- W2987364544 cites W2962834223 @default.
- W2987364544 cites W2962905471 @default.
- W2987364544 cites W2963248893 @default.
- W2987364544 cites W2963433607 @default.
- W2987364544 cites W2963470657 @default.
- W2987364544 cites W2963562522 @default.
- W2987364544 cites W2963563140 @default.
- W2987364544 cites W2963613486 @default.
- W2987364544 cites W2963698657 @default.
- W2987364544 cites W2963858717 @default.
- W2987364544 cites W2963967579 @default.
- W2987364544 cites W2964121744 @default.
- W2987364544 cites W2964210434 @default.
- W2987364544 cites W2968917279 @default.
- W2987364544 cites W2969133367 @default.
- W2987364544 cites W2971121143 @default.
- W2987364544 cites W2994747431 @default.
- W2987364544 cites W3100208521 @default.
- W2987364544 cites W3141595720 @default.
- W2987364544 cites W2741183147 @default.
- W2987364544 hasPublicationYear "2019" @default.
- W2987364544 type Work @default.
- W2987364544 sameAs 2987364544 @default.
- W2987364544 citedByCount "4" @default.
- W2987364544 countsByYear W29873645442019 @default.
- W2987364544 countsByYear W29873645442020 @default.
- W2987364544 countsByYear W29873645442022 @default.
- W2987364544 crossrefType "posted-content" @default.
- W2987364544 hasAuthorship W2987364544A5024802752 @default.
- W2987364544 hasAuthorship W2987364544A5065250840 @default.
- W2987364544 hasBestOaLocation W29873645441 @default.
- W2987364544 hasConcept C10138342 @default.
- W2987364544 hasConcept C112680207 @default.
- W2987364544 hasConcept C11413529 @default.
- W2987364544 hasConcept C126255220 @default.
- W2987364544 hasConcept C162324750 @default.
- W2987364544 hasConcept C2524010 @default.
- W2987364544 hasConcept C2777303404 @default.
- W2987364544 hasConcept C33923547 @default.
- W2987364544 hasConcept C41008148 @default.
- W2987364544 hasConcept C50522688 @default.
- W2987364544 hasConcept C60718061 @default.
- W2987364544 hasConceptScore W2987364544C10138342 @default.
- W2987364544 hasConceptScore W2987364544C112680207 @default.
- W2987364544 hasConceptScore W2987364544C11413529 @default.
- W2987364544 hasConceptScore W2987364544C126255220 @default.
- W2987364544 hasConceptScore W2987364544C162324750 @default.
- W2987364544 hasConceptScore W2987364544C2524010 @default.
- W2987364544 hasConceptScore W2987364544C2777303404 @default.
- W2987364544 hasConceptScore W2987364544C33923547 @default.
- W2987364544 hasConceptScore W2987364544C41008148 @default.
- W2987364544 hasConceptScore W2987364544C50522688 @default.
- W2987364544 hasConceptScore W2987364544C60718061 @default.
- W2987364544 hasLocation W29873645441 @default.
- W2987364544 hasLocation W29873645442 @default.
- W2987364544 hasLocation W29873645443 @default.
- W2987364544 hasLocation W29873645444 @default.
- W2987364544 hasOpenAccess W2987364544 @default.
- W2987364544 hasPrimaryLocation W29873645441 @default.