FRINK Linked Data Fragments server

Matches in SemOpenAlex for { <https://semopenalex.org/work/W3133262835> ?p ?o ?g. }

• W3133262835 abstract "Stochastic gradient algorithms are often unstable when applied to functions that do not have Lipschitz-continuous and/or bounded gradients. Gradient clipping is a simple and effective technique to stabilize the training process for problems that are prone to the exploding gradient problem. Despite its widespread popularity, the convergence properties of the gradient clipping heuristic are poorly understood, especially for stochastic problems. This paper establishes both qualitative and quantitative convergence results of the clipped stochastic (sub)gradient method (SGD) for non-smooth convex functions with rapidly growing subgradients. Our analyses show that clipping enhances the stability of SGD and that the clipped SGD algorithm enjoys finite convergence rates in many cases. We also study the convergence of a clipped method with momentum, which includes clipped SGD as a special case, for weakly convex problems under standard assumptions. With a novel Lyapunov analysis, we show that the proposed method achieves the best-known rate for the considered class of problems, demonstrating the effectiveness of clipped methods also in this regime. Numerical results confirm our theoretical developments." @default.
• W3133262835 created "2021-03-01" @default.
• W3133262835 creator A5024256519 @default.
• W3133262835 creator A5047615482 @default.
• W3133262835 date "2021-02-12" @default.
• W3133262835 modified "2023-10-01" @default.
• W3133262835 title "Stability and Convergence of Stochastic Gradient Clipping: Beyond Lipschitz Continuity and Smoothness" @default.
• W3133262835 cites W100047846 @default.
• W3133262835 cites W1553702074 @default.
• W3133262835 cites W1568288633 @default.
• W3133262835 cites W1577337550 @default.
• W3133262835 cites W1815076433 @default.
• W3133262835 cites W1972711404 @default.
• W3133262835 cites W1986324933 @default.
• W3133262835 cites W1992208280 @default.
• W3133262835 cites W2015263936 @default.
• W3133262835 cites W2019868003 @default.
• W3133262835 cites W2029008056 @default.
• W3133262835 cites W2032483793 @default.
• W3133262835 cites W2060471940 @default.
• W3133262835 cites W2061468236 @default.
• W3133262835 cites W2061657011 @default.
• W3133262835 cites W2086161653 @default.
• W3133262835 cites W2091787562 @default.
• W3133262835 cites W2156779765 @default.
• W3133262835 cites W2194775991 @default.
• W3133262835 cites W2602608495 @default.
• W3133262835 cites W2763081248 @default.
• W3133262835 cites W2894719012 @default.
• W3133262835 cites W2923133561 @default.
• W3133262835 cites W2963190258 @default.
• W3133262835 cites W2963470657 @default.
• W3133262835 cites W2964083304 @default.
• W3133262835 cites W2982573148 @default.
• W3133262835 cites W2994747431 @default.
• W3133262835 cites W3011766119 @default.
• W3133262835 cites W3025638325 @default.
• W3133262835 cites W3026800838 @default.
• W3133262835 cites W3034845664 @default.
• W3133262835 cites W3035127598 @default.
• W3133262835 cites W3035253236 @default.
• W3133262835 cites W3101450190 @default.
• W3133262835 cites W3118608800 @default.
• W3133262835 cites W836608889 @default.
• W3133262835 hasPublicationYear "2021" @default.
• W3133262835 type Work @default.
• W3133262835 sameAs 3133262835 @default.
• W3133262835 citedByCount "1" @default.
• W3133262835 countsByYear W31332628352021 @default.
• W3133262835 crossrefType "posted-content" @default.
• W3133262835 hasAuthorship W3133262835A5024256519 @default.
• W3133262835 hasAuthorship W3133262835A5047615482 @default.
• W3133262835 hasConcept C102634674 @default.
• W3133262835 hasConcept C112972136 @default.
• W3133262835 hasConcept C119857082 @default.
• W3133262835 hasConcept C126255220 @default.
• W3133262835 hasConcept C127162648 @default.
• W3133262835 hasConcept C134306372 @default.
• W3133262835 hasConcept C138885662 @default.
• W3133262835 hasConcept C154945302 @default.
• W3133262835 hasConcept C162324750 @default.
• W3133262835 hasConcept C206688291 @default.
• W3133262835 hasConcept C22324862 @default.
• W3133262835 hasConcept C2776848632 @default.
• W3133262835 hasConcept C2777303404 @default.
• W3133262835 hasConcept C28826006 @default.
• W3133262835 hasConcept C31258907 @default.
• W3133262835 hasConcept C33923547 @default.
• W3133262835 hasConcept C34388435 @default.
• W3133262835 hasConcept C41008148 @default.
• W3133262835 hasConcept C41895202 @default.
• W3133262835 hasConcept C50522688 @default.
• W3133262835 hasConcept C50644808 @default.
• W3133262835 hasConcept C57869625 @default.
• W3133262835 hasConceptScore W3133262835C102634674 @default.
• W3133262835 hasConceptScore W3133262835C112972136 @default.
• W3133262835 hasConceptScore W3133262835C119857082 @default.
• W3133262835 hasConceptScore W3133262835C126255220 @default.
• W3133262835 hasConceptScore W3133262835C127162648 @default.
• W3133262835 hasConceptScore W3133262835C134306372 @default.
• W3133262835 hasConceptScore W3133262835C138885662 @default.
• W3133262835 hasConceptScore W3133262835C154945302 @default.
• W3133262835 hasConceptScore W3133262835C162324750 @default.
• W3133262835 hasConceptScore W3133262835C206688291 @default.
• W3133262835 hasConceptScore W3133262835C22324862 @default.
• W3133262835 hasConceptScore W3133262835C2776848632 @default.
• W3133262835 hasConceptScore W3133262835C2777303404 @default.
• W3133262835 hasConceptScore W3133262835C28826006 @default.
• W3133262835 hasConceptScore W3133262835C31258907 @default.
• W3133262835 hasConceptScore W3133262835C33923547 @default.
• W3133262835 hasConceptScore W3133262835C34388435 @default.
• W3133262835 hasConceptScore W3133262835C41008148 @default.
• W3133262835 hasConceptScore W3133262835C41895202 @default.
• W3133262835 hasConceptScore W3133262835C50522688 @default.
• W3133262835 hasConceptScore W3133262835C50644808 @default.
• W3133262835 hasConceptScore W3133262835C57869625 @default.
• W3133262835 hasLocation W31332628351 @default.
• W3133262835 hasOpenAccess W3133262835 @default.
• W3133262835 hasPrimaryLocation W31332628351 @default.
• W3133262835 hasRelatedWork W1636158862 @default.

• next