Matches in SemOpenAlex for { <https://semopenalex.org/work/W3036276779> ?p ?o ?g. }
- W3036276779 abstract "This paper analyzes the trajectories of stochastic gradient descent (SGD) to help understand the algorithm's convergence properties in non-convex problems. We first show that the sequence of iterates generated by SGD remains bounded and converges with probability $1$ under a very broad range of step-size schedules. Subsequently, going beyond existing positive probability guarantees, we show that SGD avoids strict saddle points/manifolds with probability $1$ for the entire spectrum of step-size policies considered. Finally, we prove that the algorithm's rate of convergence to Hurwicz minimizers is $mathcal{O}(1/n^{p})$ if the method is employed with a $Theta(1/n^p)$ step-size schedule. This provides an important guideline for tuning the algorithm's step-size as it suggests that a cool-down phase with a vanishing step-size could lead to faster convergence; we demonstrate this heuristic using ResNet architectures on CIFAR." @default.
- W3036276779 created "2020-06-25" @default.
- W3036276779 creator A5004783895 @default.
- W3036276779 creator A5018966648 @default.
- W3036276779 creator A5027059837 @default.
- W3036276779 creator A5088619956 @default.
- W3036276779 date "2020-12-06" @default.
- W3036276779 modified "2023-10-16" @default.
- W3036276779 title "On the almost sure convergence of stochastic gradient descent in non-convex problems" @default.
- W3036276779 cites W1491706803 @default.
- W3036276779 cites W1548708049 @default.
- W3036276779 cites W1550918041 @default.
- W3036276779 cites W1557324374 @default.
- W3036276779 cites W1568229137 @default.
- W3036276779 cites W1568288633 @default.
- W3036276779 cites W1577337550 @default.
- W3036276779 cites W1965324089 @default.
- W3036276779 cites W1968360652 @default.
- W3036276779 cites W1971942712 @default.
- W3036276779 cites W1972711404 @default.
- W3036276779 cites W1987083649 @default.
- W3036276779 cites W1991667331 @default.
- W3036276779 cites W1994616650 @default.
- W3036276779 cites W2041064634 @default.
- W3036276779 cites W2060141891 @default.
- W3036276779 cites W2125812768 @default.
- W3036276779 cites W2474090883 @default.
- W3036276779 cites W2525186759 @default.
- W3036276779 cites W2531891978 @default.
- W3036276779 cites W2617691536 @default.
- W3036276779 cites W2783096200 @default.
- W3036276779 cites W2915116423 @default.
- W3036276779 cites W2962933129 @default.
- W3036276779 cites W2963092340 @default.
- W3036276779 cites W2963416883 @default.
- W3036276779 cites W2963470657 @default.
- W3036276779 cites W2964106499 @default.
- W3036276779 cites W2964121860 @default.
- W3036276779 cites W2964204240 @default.
- W3036276779 cites W2969244409 @default.
- W3036276779 cites W2970054779 @default.
- W3036276779 cites W2971250964 @default.
- W3036276779 cites W2971302301 @default.
- W3036276779 cites W2994760360 @default.
- W3036276779 cites W3141595720 @default.
- W3036276779 cites W594357522 @default.
- W3036276779 hasPublicationYear "2020" @default.
- W3036276779 type Work @default.
- W3036276779 sameAs 3036276779 @default.
- W3036276779 citedByCount "11" @default.
- W3036276779 countsByYear W30362767792020 @default.
- W3036276779 countsByYear W30362767792021 @default.
- W3036276779 crossrefType "proceedings-article" @default.
- W3036276779 hasAuthorship W3036276779A5004783895 @default.
- W3036276779 hasAuthorship W3036276779A5018966648 @default.
- W3036276779 hasAuthorship W3036276779A5027059837 @default.
- W3036276779 hasAuthorship W3036276779A5088619956 @default.
- W3036276779 hasBestOaLocation W30362767791 @default.
- W3036276779 hasConcept C112680207 @default.
- W3036276779 hasConcept C121332964 @default.
- W3036276779 hasConcept C126255220 @default.
- W3036276779 hasConcept C145446738 @default.
- W3036276779 hasConcept C153258448 @default.
- W3036276779 hasConcept C153294291 @default.
- W3036276779 hasConcept C154945302 @default.
- W3036276779 hasConcept C162324750 @default.
- W3036276779 hasConcept C206688291 @default.
- W3036276779 hasConcept C2524010 @default.
- W3036276779 hasConcept C2776637919 @default.
- W3036276779 hasConcept C2777303404 @default.
- W3036276779 hasConcept C28826006 @default.
- W3036276779 hasConcept C33923547 @default.
- W3036276779 hasConcept C41008148 @default.
- W3036276779 hasConcept C50522688 @default.
- W3036276779 hasConcept C50644808 @default.
- W3036276779 hasConceptScore W3036276779C112680207 @default.
- W3036276779 hasConceptScore W3036276779C121332964 @default.
- W3036276779 hasConceptScore W3036276779C126255220 @default.
- W3036276779 hasConceptScore W3036276779C145446738 @default.
- W3036276779 hasConceptScore W3036276779C153258448 @default.
- W3036276779 hasConceptScore W3036276779C153294291 @default.
- W3036276779 hasConceptScore W3036276779C154945302 @default.
- W3036276779 hasConceptScore W3036276779C162324750 @default.
- W3036276779 hasConceptScore W3036276779C206688291 @default.
- W3036276779 hasConceptScore W3036276779C2524010 @default.
- W3036276779 hasConceptScore W3036276779C2776637919 @default.
- W3036276779 hasConceptScore W3036276779C2777303404 @default.
- W3036276779 hasConceptScore W3036276779C28826006 @default.
- W3036276779 hasConceptScore W3036276779C33923547 @default.
- W3036276779 hasConceptScore W3036276779C41008148 @default.
- W3036276779 hasConceptScore W3036276779C50522688 @default.
- W3036276779 hasConceptScore W3036276779C50644808 @default.
- W3036276779 hasLocation W30362767791 @default.
- W3036276779 hasLocation W30362767792 @default.
- W3036276779 hasLocation W30362767793 @default.
- W3036276779 hasOpenAccess W3036276779 @default.
- W3036276779 hasPrimaryLocation W30362767791 @default.
- W3036276779 hasRelatedWork W1526607502 @default.
- W3036276779 hasRelatedWork W2570283916 @default.
- W3036276779 hasRelatedWork W2909754421 @default.