FRINK Linked Data Fragments server

Matches in SemOpenAlex for { <https://semopenalex.org/work/W3130059980> ?p ?o ?g. }

• W3130059980 abstract "SGD with Momentum (SGDM) is a widely used family of algorithms for large-scale optimization of machine learning problems. Yet, when optimizing generic convex functions, no advantage is known for any SGDM algorithm over plain SGD. Moreover, even the most recent results require changes to the SGDM algorithms, like averaging of the iterates and a projection onto a bounded domain, which are rarely used in practice. In this paper, we focus on the convergence rate of the last iterate of SGDM. For the first time, we prove that for any constant momentum factor, there exists a Lipschitz and convex function for which the last iterate of SGDM suffers from a suboptimal convergence rate of $Omega(frac{ln T}{sqrt{T}})$ after $T$ iterations. Based on this fact, we study a class of (both adaptive and non-adaptive) Follow-The-Regularized-Leader-based SGDM algorithms with increasing momentum and shrinking updates. For these algorithms, we show that the last iterate has optimal convergence $O(frac{1}{sqrt{T}})$ for unconstrained convex stochastic optimization problems without projections onto bounded domains nor knowledge of $T$. Further, we show a variety of results for FTRL-based SGDM when used with adaptive stepsizes. Empirical results are shown as well." @default.
• W3130059980 created "2021-03-01" @default.
• W3130059980 creator A5044260265 @default.
• W3130059980 creator A5064437230 @default.
• W3130059980 creator A5082020327 @default.
• W3130059980 date "2021-02-13" @default.
• W3130059980 modified "2023-10-16" @default.
• W3130059980 title "On the Last Iterate Convergence of Momentum Methods" @default.
• W3130059980 cites W104184427 @default.
• W3130059980 cites W1505731132 @default.
• W3130059980 cites W1508384000 @default.
• W3130059980 cites W1518461880 @default.
• W3130059980 cites W1988720110 @default.
• W3130059980 cites W2090852255 @default.
• W3130059980 cites W2096840748 @default.
• W3130059980 cites W2118075434 @default.
• W3130059980 cites W2146502635 @default.
• W3130059980 cites W2153635508 @default.
• W3130059980 cites W2162287622 @default.
• W3130059980 cites W2169713291 @default.
• W3130059980 cites W2337540838 @default.
• W3130059980 cites W2785523195 @default.
• W3130059980 cites W2896146108 @default.
• W3130059980 cites W2907225497 @default.
• W3130059980 cites W2945983450 @default.
• W3130059980 cites W2946511237 @default.
• W3130059980 cites W2962712496 @default.
• W3130059980 cites W2963613486 @default.
• W3130059980 cites W2963834323 @default.
• W3130059980 cites W2964066689 @default.
• W3130059980 cites W2964121744 @default.
• W3130059980 cites W2965944281 @default.
• W3130059980 cites W2970623506 @default.
• W3130059980 cites W2997976910 @default.
• W3130059980 cites W3035358246 @default.
• W3130059980 cites W3090356302 @default.
• W3130059980 cites W3093856905 @default.
• W3130059980 cites W3121837399 @default.
• W3130059980 cites W3167256008 @default.
• W3130059980 cites W6657555 @default.
• W3130059980 doi "https://doi.org/10.48550/arxiv.2102.07002" @default.
• W3130059980 hasPublicationYear "2021" @default.
• W3130059980 type Work @default.
• W3130059980 sameAs 3130059980 @default.
• W3130059980 citedByCount "0" @default.
• W3130059980 crossrefType "posted-content" @default.
• W3130059980 hasAuthorship W3130059980A5044260265 @default.
• W3130059980 hasAuthorship W3130059980A5064437230 @default.
• W3130059980 hasAuthorship W3130059980A5082020327 @default.
• W3130059980 hasBestOaLocation W31300599801 @default.
• W3130059980 hasConcept C10138342 @default.
• W3130059980 hasConcept C112680207 @default.
• W3130059980 hasConcept C11413529 @default.
• W3130059980 hasConcept C118615104 @default.
• W3130059980 hasConcept C121332964 @default.
• W3130059980 hasConcept C126255220 @default.
• W3130059980 hasConcept C134306372 @default.
• W3130059980 hasConcept C14036430 @default.
• W3130059980 hasConcept C140479938 @default.
• W3130059980 hasConcept C145446738 @default.
• W3130059980 hasConcept C157972887 @default.
• W3130059980 hasConcept C162324750 @default.
• W3130059980 hasConcept C202444582 @default.
• W3130059980 hasConcept C22324862 @default.
• W3130059980 hasConcept C2524010 @default.
• W3130059980 hasConcept C26517878 @default.
• W3130059980 hasConcept C2777303404 @default.
• W3130059980 hasConcept C2779557605 @default.
• W3130059980 hasConcept C28826006 @default.
• W3130059980 hasConcept C33923547 @default.
• W3130059980 hasConcept C34388435 @default.
• W3130059980 hasConcept C36503486 @default.
• W3130059980 hasConcept C38652104 @default.
• W3130059980 hasConcept C41008148 @default.
• W3130059980 hasConcept C50522688 @default.
• W3130059980 hasConcept C57493831 @default.
• W3130059980 hasConcept C57869625 @default.
• W3130059980 hasConcept C60718061 @default.
• W3130059980 hasConcept C62520636 @default.
• W3130059980 hasConcept C78458016 @default.
• W3130059980 hasConcept C86803240 @default.
• W3130059980 hasConceptScore W3130059980C10138342 @default.
• W3130059980 hasConceptScore W3130059980C112680207 @default.
• W3130059980 hasConceptScore W3130059980C11413529 @default.
• W3130059980 hasConceptScore W3130059980C118615104 @default.
• W3130059980 hasConceptScore W3130059980C121332964 @default.
• W3130059980 hasConceptScore W3130059980C126255220 @default.
• W3130059980 hasConceptScore W3130059980C134306372 @default.
• W3130059980 hasConceptScore W3130059980C14036430 @default.
• W3130059980 hasConceptScore W3130059980C140479938 @default.
• W3130059980 hasConceptScore W3130059980C145446738 @default.
• W3130059980 hasConceptScore W3130059980C157972887 @default.
• W3130059980 hasConceptScore W3130059980C162324750 @default.
• W3130059980 hasConceptScore W3130059980C202444582 @default.
• W3130059980 hasConceptScore W3130059980C22324862 @default.
• W3130059980 hasConceptScore W3130059980C2524010 @default.
• W3130059980 hasConceptScore W3130059980C26517878 @default.
• W3130059980 hasConceptScore W3130059980C2777303404 @default.
• W3130059980 hasConceptScore W3130059980C2779557605 @default.
• W3130059980 hasConceptScore W3130059980C28826006 @default.

• next