FRINK Linked Data Fragments server

Matches in SemOpenAlex for { <https://semopenalex.org/work/W3158974429> ?p ?o ?g. }

• W3158974429 abstract "In this paper, we demonstrate the power of a widely used stochastic estimator based on moving average (SEMA) on a range of stochastic non-convex optimization problems, which only requires {bf a general unbiased stochastic oracle}. We analyze various stochastic methods (existing or newly proposed) based on the {bf variance recursion property} of SEMA for three families of non-convex optimization, namely standard stochastic non-convex minimization, stochastic non-convex strongly-concave min-max optimization, and stochastic bilevel optimization. Our contributions include: (i) for standard stochastic non-convex minimization, we present a simple and intuitive proof of convergence for a family Adam-style methods (including Adam) with an increasing or large momentum parameter for the first-order moment, which gives an alternative yet more natural way to guarantee Adam converge; (ii) for stochastic non-convex strongly-concave min-max optimization, we present a single-loop stochastic gradient descent ascent method based on the moving average estimators and establish its oracle complexity of $O(1/epsilon^4)$ without using a large mini-batch size, addressing a gap in the literature; (iii) for stochastic bilevel optimization, we present a single-loop stochastic method based on the moving average estimators and establish its oracle complexity of $widetilde O(1/epsilon^4)$ without computing the inverse or SVD of the Hessian matrix, improving state-of-the-art results. For all these problems, we also establish a variance diminishing result for the used stochastic gradient estimators." @default.
• W3158974429 created "2021-05-10" @default.
• W3158974429 creator A5018343158 @default.
• W3158974429 creator A5023288846 @default.
• W3158974429 creator A5054429972 @default.
• W3158974429 creator A5069394608 @default.
• W3158974429 creator A5085908411 @default.
• W3158974429 date "2021-04-30" @default.
• W3158974429 modified "2023-09-27" @default.
• W3158974429 title "On Stochastic Moving-Average Estimators for Non-Convex Optimization." @default.
• W3158974429 cites W1494085563 @default.
• W3158974429 cites W1575658237 @default.
• W3158974429 cites W2082261506 @default.
• W3158974429 cites W2146502635 @default.
• W3158974429 cites W2263490141 @default.
• W3158974429 cites W2337540838 @default.
• W3158974429 cites W2741269719 @default.
• W3158974429 cites W2785523195 @default.
• W3158974429 cites W2787415863 @default.
• W3158974429 cites W2803444541 @default.
• W3158974429 cites W2810075754 @default.
• W3158974429 cites W2893812806 @default.
• W3158974429 cites W2907225497 @default.
• W3158974429 cites W2944542720 @default.
• W3158974429 cites W2946511237 @default.
• W3158974429 cites W2947578309 @default.
• W3158974429 cites W2963306862 @default.
• W3158974429 cites W2963540381 @default.
• W3158974429 cites W2963563140 @default.
• W3158974429 cites W2963613486 @default.
• W3158974429 cites W2963698657 @default.
• W3158974429 cites W2964121744 @default.
• W3158974429 cites W2964158744 @default.
• W3158974429 cites W2970697704 @default.
• W3158974429 cites W2993258424 @default.
• W3158974429 cites W2994689640 @default.
• W3158974429 cites W3009948090 @default.
• W3158974429 cites W3011766119 @default.
• W3158974429 cites W3034995656 @default.
• W3158974429 cites W3035253236 @default.
• W3158974429 cites W3035278189 @default.
• W3158974429 cites W3039770199 @default.
• W3158974429 cites W3041129870 @default.
• W3158974429 cites W3044458766 @default.
• W3158974429 cites W3054970827 @default.
• W3158974429 cites W3092809425 @default.
• W3158974429 cites W3096312061 @default.
• W3158974429 cites W3099928128 @default.
• W3158974429 cites W3101511375 @default.
• W3158974429 cites W3102608064 @default.
• W3158974429 cites W3102802540 @default.
• W3158974429 cites W3110015897 @default.
• W3158974429 cites W3112047150 @default.
• W3158974429 cites W3127438322 @default.
• W3158974429 cites W3134825596 @default.
• W3158974429 cites W3135167421 @default.
• W3158974429 cites W3151497657 @default.
• W3158974429 hasPublicationYear "2021" @default.
• W3158974429 type Work @default.
• W3158974429 sameAs 3158974429 @default.
• W3158974429 citedByCount "8" @default.
• W3158974429 countsByYear W31589744292021 @default.
• W3158974429 crossrefType "posted-content" @default.
• W3158974429 hasAuthorship W3158974429A5018343158 @default.
• W3158974429 hasAuthorship W3158974429A5023288846 @default.
• W3158974429 hasAuthorship W3158974429A5054429972 @default.
• W3158974429 hasAuthorship W3158974429A5069394608 @default.
• W3158974429 hasAuthorship W3158974429A5085908411 @default.
• W3158974429 hasConcept C105795698 @default.
• W3158974429 hasConcept C112680207 @default.
• W3158974429 hasConcept C115903868 @default.
• W3158974429 hasConcept C119857082 @default.
• W3158974429 hasConcept C126255220 @default.
• W3158974429 hasConcept C157972887 @default.
• W3158974429 hasConcept C185429906 @default.
• W3158974429 hasConcept C194387892 @default.
• W3158974429 hasConcept C206688291 @default.
• W3158974429 hasConcept C2524010 @default.
• W3158974429 hasConcept C26517878 @default.
• W3158974429 hasConcept C28826006 @default.
• W3158974429 hasConcept C33923547 @default.
• W3158974429 hasConcept C38652104 @default.
• W3158974429 hasConcept C41008148 @default.
• W3158974429 hasConcept C50644808 @default.
• W3158974429 hasConcept C55166926 @default.
• W3158974429 hasConcept C55479107 @default.
• W3158974429 hasConceptScore W3158974429C105795698 @default.
• W3158974429 hasConceptScore W3158974429C112680207 @default.
• W3158974429 hasConceptScore W3158974429C115903868 @default.
• W3158974429 hasConceptScore W3158974429C119857082 @default.
• W3158974429 hasConceptScore W3158974429C126255220 @default.
• W3158974429 hasConceptScore W3158974429C157972887 @default.
• W3158974429 hasConceptScore W3158974429C185429906 @default.
• W3158974429 hasConceptScore W3158974429C194387892 @default.
• W3158974429 hasConceptScore W3158974429C206688291 @default.
• W3158974429 hasConceptScore W3158974429C2524010 @default.
• W3158974429 hasConceptScore W3158974429C26517878 @default.
• W3158974429 hasConceptScore W3158974429C28826006 @default.
• W3158974429 hasConceptScore W3158974429C33923547 @default.
• W3158974429 hasConceptScore W3158974429C38652104 @default.

• next