Matches in SemOpenAlex for { <https://semopenalex.org/work/W4375957515> ?p ?o ?g. }
Showing items 1 to 71 of
71
with 100 items per page.
- W4375957515 abstract "In this paper, we present a comprehensive study on the convergence properties of Adam-family methods for nonsmooth optimization, especially in the training of nonsmooth neural networks. We introduce a novel two-timescale framework that adopts a two-timescale updating scheme, and prove its convergence properties under mild assumptions. Our proposed framework encompasses various popular Adam-family methods, providing convergence guarantees for these methods in training nonsmooth neural networks. Furthermore, we develop stochastic subgradient methods that incorporate gradient clipping techniques for training nonsmooth neural networks with heavy-tailed noise. Through our framework, we show that our proposed methods converge even when the evaluation noises are only assumed to be integrable. Extensive numerical experiments demonstrate the high efficiency and robustness of our proposed methods." @default.
- W4375957515 created "2023-05-10" @default.
- W4375957515 creator A5028136951 @default.
- W4375957515 creator A5057647276 @default.
- W4375957515 creator A5062240635 @default.
- W4375957515 creator A5064083862 @default.
- W4375957515 date "2023-05-06" @default.
- W4375957515 modified "2023-09-30" @default.
- W4375957515 title "Adam-family Methods for Nonsmooth Optimization with Convergence Guarantees" @default.
- W4375957515 doi "https://doi.org/10.48550/arxiv.2305.03938" @default.
- W4375957515 hasPublicationYear "2023" @default.
- W4375957515 type Work @default.
- W4375957515 citedByCount "0" @default.
- W4375957515 crossrefType "posted-content" @default.
- W4375957515 hasAuthorship W4375957515A5028136951 @default.
- W4375957515 hasAuthorship W4375957515A5057647276 @default.
- W4375957515 hasAuthorship W4375957515A5062240635 @default.
- W4375957515 hasAuthorship W4375957515A5064083862 @default.
- W4375957515 hasBestOaLocation W43759575151 @default.
- W4375957515 hasConcept C104317684 @default.
- W4375957515 hasConcept C126255220 @default.
- W4375957515 hasConcept C134306372 @default.
- W4375957515 hasConcept C138885662 @default.
- W4375957515 hasConcept C154945302 @default.
- W4375957515 hasConcept C158968445 @default.
- W4375957515 hasConcept C162324750 @default.
- W4375957515 hasConcept C185592680 @default.
- W4375957515 hasConcept C2776848632 @default.
- W4375957515 hasConcept C2777303404 @default.
- W4375957515 hasConcept C33923547 @default.
- W4375957515 hasConcept C41008148 @default.
- W4375957515 hasConcept C41895202 @default.
- W4375957515 hasConcept C50522688 @default.
- W4375957515 hasConcept C50644808 @default.
- W4375957515 hasConcept C55493867 @default.
- W4375957515 hasConcept C63479239 @default.
- W4375957515 hasConcept C77618280 @default.
- W4375957515 hasConceptScore W4375957515C104317684 @default.
- W4375957515 hasConceptScore W4375957515C126255220 @default.
- W4375957515 hasConceptScore W4375957515C134306372 @default.
- W4375957515 hasConceptScore W4375957515C138885662 @default.
- W4375957515 hasConceptScore W4375957515C154945302 @default.
- W4375957515 hasConceptScore W4375957515C158968445 @default.
- W4375957515 hasConceptScore W4375957515C162324750 @default.
- W4375957515 hasConceptScore W4375957515C185592680 @default.
- W4375957515 hasConceptScore W4375957515C2776848632 @default.
- W4375957515 hasConceptScore W4375957515C2777303404 @default.
- W4375957515 hasConceptScore W4375957515C33923547 @default.
- W4375957515 hasConceptScore W4375957515C41008148 @default.
- W4375957515 hasConceptScore W4375957515C41895202 @default.
- W4375957515 hasConceptScore W4375957515C50522688 @default.
- W4375957515 hasConceptScore W4375957515C50644808 @default.
- W4375957515 hasConceptScore W4375957515C55493867 @default.
- W4375957515 hasConceptScore W4375957515C63479239 @default.
- W4375957515 hasConceptScore W4375957515C77618280 @default.
- W4375957515 hasLocation W43759575151 @default.
- W4375957515 hasOpenAccess W4375957515 @default.
- W4375957515 hasPrimaryLocation W43759575151 @default.
- W4375957515 hasRelatedWork W1963589315 @default.
- W4375957515 hasRelatedWork W2062919631 @default.
- W4375957515 hasRelatedWork W2064986600 @default.
- W4375957515 hasRelatedWork W2092695897 @default.
- W4375957515 hasRelatedWork W2127198104 @default.
- W4375957515 hasRelatedWork W2314329970 @default.
- W4375957515 hasRelatedWork W2355978539 @default.
- W4375957515 hasRelatedWork W2363143319 @default.
- W4375957515 hasRelatedWork W2757616806 @default.
- W4375957515 hasRelatedWork W2095293854 @default.
- W4375957515 isParatext "false" @default.
- W4375957515 isRetracted "false" @default.
- W4375957515 workType "article" @default.