FRINK Linked Data Fragments server

Matches in SemOpenAlex for { <https://semopenalex.org/work/W3132776989> ?p ?o ?g. }

• W3132776989 abstract "The convergence of stochastic gradient descent is highly dependent on the step-size, especially on non-convex problems such as neural network training. Step decay step-size schedules (constant and then cut) are widely used in practice because of their excellent convergence and generalization qualities, but their theoretical properties are not yet well understood. We provide the convergence results for step decay in the non-convex regime, ensuring that the gradient norm vanishes at an $mathcal{O}(ln T/sqrt{T})$ rate. We also provide the convergence guarantees for general (possibly non-smooth) convex problems, ensuring an $mathcal{O}(ln T/sqrt{T})$ convergence rate. Finally, in the strongly convex case, we establish an $mathcal{O}(ln T/T)$ rate for smooth problems, which we also prove to be tight, and an $mathcal{O}(ln^2 T /T)$ rate without the smoothness assumption. We illustrate the practical efficiency of the step decay step-size in several large scale deep neural network training tasks." @default.
• W3132776989 created "2021-03-01" @default.
• W3132776989 creator A5024256519 @default.
• W3132776989 creator A5024792434 @default.
• W3132776989 creator A5054704457 @default.
• W3132776989 date "2021-02-18" @default.
• W3132776989 modified "2023-10-03" @default.
• W3132776989 title "On the Convergence of Step Decay Step-Size for Stochastic Optimization" @default.
• W3132776989 cites W104184427 @default.
• W3132776989 cites W1516903196 @default.
• W3132776989 cites W1962291245 @default.
• W3132776989 cites W1992208280 @default.
• W3132776989 cites W1994616650 @default.
• W3132776989 cites W2008164266 @default.
• W3132776989 cites W2029463628 @default.
• W3132776989 cites W2044207982 @default.
• W3132776989 cites W2112269233 @default.
• W3132776989 cites W2146502635 @default.
• W3132776989 cites W2151695970 @default.
• W3132776989 cites W2154682027 @default.
• W3132776989 cites W2156779765 @default.
• W3132776989 cites W2163605009 @default.
• W3132776989 cites W2194775991 @default.
• W3132776989 cites W2271840356 @default.
• W3132776989 cites W2559932663 @default.
• W3132776989 cites W2899771611 @default.
• W3132776989 cites W2908510526 @default.
• W3132776989 cites W2962883515 @default.
• W3132776989 cites W2963446712 @default.
• W3132776989 cites W2963470657 @default.
• W3132776989 cites W2964083304 @default.
• W3132776989 cites W2964121744 @default.
• W3132776989 cites W2964725673 @default.
• W3132776989 cites W2965944281 @default.
• W3132776989 cites W2970028551 @default.
• W3132776989 cites W2970038219 @default.
• W3132776989 cites W2970498991 @default.
• W3132776989 cites W2974142167 @default.
• W3132776989 cites W2980686465 @default.
• W3132776989 cites W2990219166 @default.
• W3132776989 cites W3006996801 @default.
• W3132776989 cites W3007118755 @default.
• W3132776989 cites W3035161196 @default.
• W3132776989 cites W3141595720 @default.
• W3132776989 cites W3169624762 @default.
• W3132776989 cites W3094251777 @default.
• W3132776989 hasPublicationYear "2021" @default.
• W3132776989 type Work @default.
• W3132776989 sameAs 3132776989 @default.
• W3132776989 citedByCount "1" @default.
• W3132776989 countsByYear W31327769892021 @default.
• W3132776989 crossrefType "posted-content" @default.
• W3132776989 hasAuthorship W3132776989A5024256519 @default.
• W3132776989 hasAuthorship W3132776989A5024792434 @default.
• W3132776989 hasAuthorship W3132776989A5054704457 @default.
• W3132776989 hasConcept C102634674 @default.
• W3132776989 hasConcept C112680207 @default.
• W3132776989 hasConcept C114614502 @default.
• W3132776989 hasConcept C118615104 @default.
• W3132776989 hasConcept C127162648 @default.
• W3132776989 hasConcept C134306372 @default.
• W3132776989 hasConcept C145446738 @default.
• W3132776989 hasConcept C153258448 @default.
• W3132776989 hasConcept C154945302 @default.
• W3132776989 hasConcept C162324750 @default.
• W3132776989 hasConcept C17744445 @default.
• W3132776989 hasConcept C191795146 @default.
• W3132776989 hasConcept C199539241 @default.
• W3132776989 hasConcept C206688291 @default.
• W3132776989 hasConcept C2524010 @default.
• W3132776989 hasConcept C2777303404 @default.
• W3132776989 hasConcept C28826006 @default.
• W3132776989 hasConcept C31258907 @default.
• W3132776989 hasConcept C33923547 @default.
• W3132776989 hasConcept C41008148 @default.
• W3132776989 hasConcept C50522688 @default.
• W3132776989 hasConcept C50644808 @default.
• W3132776989 hasConcept C57869625 @default.
• W3132776989 hasConceptScore W3132776989C102634674 @default.
• W3132776989 hasConceptScore W3132776989C112680207 @default.
• W3132776989 hasConceptScore W3132776989C114614502 @default.
• W3132776989 hasConceptScore W3132776989C118615104 @default.
• W3132776989 hasConceptScore W3132776989C127162648 @default.
• W3132776989 hasConceptScore W3132776989C134306372 @default.
• W3132776989 hasConceptScore W3132776989C145446738 @default.
• W3132776989 hasConceptScore W3132776989C153258448 @default.
• W3132776989 hasConceptScore W3132776989C154945302 @default.
• W3132776989 hasConceptScore W3132776989C162324750 @default.
• W3132776989 hasConceptScore W3132776989C17744445 @default.
• W3132776989 hasConceptScore W3132776989C191795146 @default.
• W3132776989 hasConceptScore W3132776989C199539241 @default.
• W3132776989 hasConceptScore W3132776989C206688291 @default.
• W3132776989 hasConceptScore W3132776989C2524010 @default.
• W3132776989 hasConceptScore W3132776989C2777303404 @default.
• W3132776989 hasConceptScore W3132776989C28826006 @default.
• W3132776989 hasConceptScore W3132776989C31258907 @default.
• W3132776989 hasConceptScore W3132776989C33923547 @default.
• W3132776989 hasConceptScore W3132776989C41008148 @default.
• W3132776989 hasConceptScore W3132776989C50522688 @default.
• W3132776989 hasConceptScore W3132776989C50644808 @default.

• next