Matches in SemOpenAlex for { <https://semopenalex.org/work/W4384919883> ?p ?o ?g. }
Showing items 1 to 75 of
75
with 100 items per page.
- W4384919883 abstract "In this paper, we investigate the convergence properties of the stochastic gradient descent (SGD) method and its variants, especially in training neural networks built from nonsmooth activation functions. We develop a novel framework that assigns different timescales to stepsizes for updating the momentum terms and variables, respectively. Under mild conditions, we prove the global convergence of our proposed framework in both single-timescale and two-timescale cases. We show that our proposed framework encompasses a wide range of well-known SGD-type methods, including heavy-ball SGD, SignSGD, Lion, normalized SGD and clipped SGD. Furthermore, when the objective function adopts a finite-sum formulation, we prove the convergence properties for these SGD-type methods based on our proposed framework. In particular, we prove that these SGD-type methods find the Clarke stationary points of the objective function with randomly chosen stepsizes and initial points under mild assumptions. Preliminary numerical experiments demonstrate the high efficiency of our analyzed SGD-type methods." @default.
- W4384919883 created "2023-07-21" @default.
- W4384919883 creator A5028136951 @default.
- W4384919883 creator A5062240635 @default.
- W4384919883 creator A5064083862 @default.
- W4384919883 date "2023-07-19" @default.
- W4384919883 modified "2023-09-27" @default.
- W4384919883 title "Convergence Guarantees for Stochastic Subgradient Methods in Nonsmooth Nonconvex Optimization" @default.
- W4384919883 doi "https://doi.org/10.48550/arxiv.2307.10053" @default.
- W4384919883 hasPublicationYear "2023" @default.
- W4384919883 type Work @default.
- W4384919883 citedByCount "0" @default.
- W4384919883 crossrefType "posted-content" @default.
- W4384919883 hasAuthorship W4384919883A5028136951 @default.
- W4384919883 hasAuthorship W4384919883A5062240635 @default.
- W4384919883 hasAuthorship W4384919883A5064083862 @default.
- W4384919883 hasBestOaLocation W43849198831 @default.
- W4384919883 hasConcept C126255220 @default.
- W4384919883 hasConcept C134306372 @default.
- W4384919883 hasConcept C14036430 @default.
- W4384919883 hasConcept C154945302 @default.
- W4384919883 hasConcept C158968445 @default.
- W4384919883 hasConcept C159985019 @default.
- W4384919883 hasConcept C162324750 @default.
- W4384919883 hasConcept C18903297 @default.
- W4384919883 hasConcept C189237950 @default.
- W4384919883 hasConcept C192562407 @default.
- W4384919883 hasConcept C204323151 @default.
- W4384919883 hasConcept C206688291 @default.
- W4384919883 hasConcept C2777299769 @default.
- W4384919883 hasConcept C2777303404 @default.
- W4384919883 hasConcept C28826006 @default.
- W4384919883 hasConcept C33923547 @default.
- W4384919883 hasConcept C41008148 @default.
- W4384919883 hasConcept C50522688 @default.
- W4384919883 hasConcept C50644808 @default.
- W4384919883 hasConcept C78458016 @default.
- W4384919883 hasConcept C86803240 @default.
- W4384919883 hasConceptScore W4384919883C126255220 @default.
- W4384919883 hasConceptScore W4384919883C134306372 @default.
- W4384919883 hasConceptScore W4384919883C14036430 @default.
- W4384919883 hasConceptScore W4384919883C154945302 @default.
- W4384919883 hasConceptScore W4384919883C158968445 @default.
- W4384919883 hasConceptScore W4384919883C159985019 @default.
- W4384919883 hasConceptScore W4384919883C162324750 @default.
- W4384919883 hasConceptScore W4384919883C18903297 @default.
- W4384919883 hasConceptScore W4384919883C189237950 @default.
- W4384919883 hasConceptScore W4384919883C192562407 @default.
- W4384919883 hasConceptScore W4384919883C204323151 @default.
- W4384919883 hasConceptScore W4384919883C206688291 @default.
- W4384919883 hasConceptScore W4384919883C2777299769 @default.
- W4384919883 hasConceptScore W4384919883C2777303404 @default.
- W4384919883 hasConceptScore W4384919883C28826006 @default.
- W4384919883 hasConceptScore W4384919883C33923547 @default.
- W4384919883 hasConceptScore W4384919883C41008148 @default.
- W4384919883 hasConceptScore W4384919883C50522688 @default.
- W4384919883 hasConceptScore W4384919883C50644808 @default.
- W4384919883 hasConceptScore W4384919883C78458016 @default.
- W4384919883 hasConceptScore W4384919883C86803240 @default.
- W4384919883 hasLocation W43849198831 @default.
- W4384919883 hasOpenAccess W4384919883 @default.
- W4384919883 hasPrimaryLocation W43849198831 @default.
- W4384919883 hasRelatedWork W1963589315 @default.
- W4384919883 hasRelatedWork W2055039565 @default.
- W4384919883 hasRelatedWork W2088312778 @default.
- W4384919883 hasRelatedWork W2314329970 @default.
- W4384919883 hasRelatedWork W2363143319 @default.
- W4384919883 hasRelatedWork W2757616806 @default.
- W4384919883 hasRelatedWork W2807821938 @default.
- W4384919883 hasRelatedWork W4309582248 @default.
- W4384919883 hasRelatedWork W4384919883 @default.
- W4384919883 hasRelatedWork W2095293854 @default.
- W4384919883 isParatext "false" @default.
- W4384919883 isRetracted "false" @default.
- W4384919883 workType "article" @default.