FRINK Linked Data Fragments server

Matches in SemOpenAlex for { <https://semopenalex.org/work/W3110931384> ?p ?o ?g. }

• W3110931384 endingPage "1118" @default.
• W3110931384 startingPage "1107" @default.
• W3110931384 abstract "Momentum technique has recently emerged as an effective strategy in accelerating convergence of gradient descent (GD) methods and exhibits improved performance in deep learning as well as regularized learning. Typical momentum examples include Nesterov's accelerated gradient (NAG) and heavy-ball (HB) methods. However, so far, almost all the acceleration analyses are only limited to NAG, and a few investigations about the acceleration of HB are reported. In this article, we address the convergence about the last iterate of HB in nonsmooth optimizations with constraints, which we name individual convergence. This question is significant in machine learning, where the constraints are required to impose on the learning structure and the individual output is needed to effectively guarantee this structure while keeping an optimal rate of convergence. Specifically, we prove that HB achieves an individual convergence rate of O(1/√t) , where t is the number of iterations. This indicates that both of the two momentum methods can accelerate the individual convergence of basic GD to be optimal. Even for the convergence of averaged iterates, our result avoids the disadvantages of the previous work in restricting the optimization problem to be unconstrained as well as limiting the performed number of iterations to be predefined. The novelty of convergence analysis presented in this article provides a clear understanding of how the HB momentum can accelerate the individual convergence and reveals more insights about the similarities and differences in getting the averaging and individual convergence rates. The derived optimal individual convergence is extended to regularized and stochastic settings, in which an individual solution can be produced by the projection-based operation. In contrast to the averaged output, the sparsity can be reduced remarkably without sacrificing the theoretical optimal rates. Several real experiments demonstrate the performance of HB momentum strategy." @default.
• W3110931384 created "2020-12-21" @default.
• W3110931384 creator A5046355512 @default.
• W3110931384 creator A5052908126 @default.
• W3110931384 creator A5053102123 @default.
• W3110931384 date "2022-03-01" @default.
• W3110931384 modified "2023-10-16" @default.
• W3110931384 title "Momentum Acceleration in the Individual Convergence of Nonsmooth Convex Optimization With Constraints" @default.
• W3110931384 cites W1988720110 @default.
• W3110931384 cites W2000462146 @default.
• W3110931384 cites W2024484010 @default.
• W3110931384 cites W2028148084 @default.
• W3110931384 cites W2090852255 @default.
• W3110931384 cites W2091825929 @default.
• W3110931384 cites W2101868363 @default.
• W3110931384 cites W2115807064 @default.
• W3110931384 cites W2124768887 @default.
• W3110931384 cites W2129160848 @default.
• W3110931384 cites W2137731592 @default.
• W3110931384 cites W2142623206 @default.
• W3110931384 cites W2217490359 @default.
• W3110931384 cites W2413731729 @default.
• W3110931384 cites W2896146108 @default.
• W3110931384 cites W2912848192 @default.
• W3110931384 cites W2923133561 @default.
• W3110931384 cites W2963433607 @default.
• W3110931384 doi "https://doi.org/10.1109/tnnls.2020.3040325" @default.
• W3110931384 hasPubMedId "https://pubmed.ncbi.nlm.nih.gov/33290233" @default.
• W3110931384 hasPublicationYear "2022" @default.
• W3110931384 type Work @default.
• W3110931384 sameAs 3110931384 @default.
• W3110931384 citedByCount "5" @default.
• W3110931384 countsByYear W31109313842022 @default.
• W3110931384 countsByYear W31109313842023 @default.
• W3110931384 crossrefType "journal-article" @default.
• W3110931384 hasAuthorship W3110931384A5046355512 @default.
• W3110931384 hasAuthorship W3110931384A5052908126 @default.
• W3110931384 hasAuthorship W3110931384A5053102123 @default.
• W3110931384 hasConcept C10138342 @default.
• W3110931384 hasConcept C112680207 @default.
• W3110931384 hasConcept C117896860 @default.
• W3110931384 hasConcept C121332964 @default.
• W3110931384 hasConcept C126255220 @default.
• W3110931384 hasConcept C127162648 @default.
• W3110931384 hasConcept C134306372 @default.
• W3110931384 hasConcept C140479938 @default.
• W3110931384 hasConcept C145446738 @default.
• W3110931384 hasConcept C162324750 @default.
• W3110931384 hasConcept C2524010 @default.
• W3110931384 hasConcept C2777303404 @default.
• W3110931384 hasConcept C28826006 @default.
• W3110931384 hasConcept C31258907 @default.
• W3110931384 hasConcept C33923547 @default.
• W3110931384 hasConcept C41008148 @default.
• W3110931384 hasConcept C50522688 @default.
• W3110931384 hasConcept C57869625 @default.
• W3110931384 hasConcept C60718061 @default.
• W3110931384 hasConcept C74650414 @default.
• W3110931384 hasConceptScore W3110931384C10138342 @default.
• W3110931384 hasConceptScore W3110931384C112680207 @default.
• W3110931384 hasConceptScore W3110931384C117896860 @default.
• W3110931384 hasConceptScore W3110931384C121332964 @default.
• W3110931384 hasConceptScore W3110931384C126255220 @default.
• W3110931384 hasConceptScore W3110931384C127162648 @default.
• W3110931384 hasConceptScore W3110931384C134306372 @default.
• W3110931384 hasConceptScore W3110931384C140479938 @default.
• W3110931384 hasConceptScore W3110931384C145446738 @default.
• W3110931384 hasConceptScore W3110931384C162324750 @default.
• W3110931384 hasConceptScore W3110931384C2524010 @default.
• W3110931384 hasConceptScore W3110931384C2777303404 @default.
• W3110931384 hasConceptScore W3110931384C28826006 @default.
• W3110931384 hasConceptScore W3110931384C31258907 @default.
• W3110931384 hasConceptScore W3110931384C33923547 @default.
• W3110931384 hasConceptScore W3110931384C41008148 @default.
• W3110931384 hasConceptScore W3110931384C50522688 @default.
• W3110931384 hasConceptScore W3110931384C57869625 @default.
• W3110931384 hasConceptScore W3110931384C60718061 @default.
• W3110931384 hasConceptScore W3110931384C74650414 @default.
• W3110931384 hasFunder F4320321001 @default.
• W3110931384 hasIssue "3" @default.
• W3110931384 hasLocation W31109313841 @default.
• W3110931384 hasLocation W31109313842 @default.
• W3110931384 hasOpenAccess W3110931384 @default.
• W3110931384 hasPrimaryLocation W31109313841 @default.
• W3110931384 hasRelatedWork W200242438 @default.
• W3110931384 hasRelatedWork W2038589392 @default.
• W3110931384 hasRelatedWork W2137731592 @default.
• W3110931384 hasRelatedWork W2952048857 @default.
• W3110931384 hasRelatedWork W2952586095 @default.
• W3110931384 hasRelatedWork W2985014567 @default.
• W3110931384 hasRelatedWork W3038912686 @default.
• W3110931384 hasRelatedWork W4298016328 @default.
• W3110931384 hasRelatedWork W4298668733 @default.
• W3110931384 hasRelatedWork W4324119293 @default.
• W3110931384 hasVolume "33" @default.
• W3110931384 isParatext "false" @default.
• W3110931384 isRetracted "false" @default.
• W3110931384 magId "3110931384" @default.

• next