Matches in SemOpenAlex for { <https://semopenalex.org/work/W4295957051> ?p ?o ?g. }
Showing items 1 to 81 of
81
with 100 items per page.
- W4295957051 abstract "First-order stochastic methods for solving large-scale non-convex optimization problems are widely used in many big-data applications, e.g. training deep neural networks as well as other complex and potentially non-convex machine learning models. Their inexpensive iterations generally come together with slow global convergence rate (mostly sublinear), leading to the necessity of carrying out a very high number of iterations before the iterates reach a neighborhood of a minimizer. In this work, we present a first-order stochastic algorithm based on a combination of homotopy methods and SGD, called Homotopy-Stochastic Gradient Descent (H-SGD), which finds interesting connections with some proposed heuristics in the literature, e.g. optimization by Gaussian continuation, training by diffusion, mollifying networks. Under some mild assumptions on the problem structure, we conduct a theoretical analysis of the proposed algorithm. Our analysis shows that, with a specifically designed scheme for the homotopy parameter, H-SGD enjoys a global linear rate of convergence to a neighborhood of a minimum while maintaining fast and inexpensive iterations. Experimental evaluations confirm the theoretical results and show that H-SGD can outperform standard SGD." @default.
- W4295957051 created "2022-09-16" @default.
- W4295957051 creator A5031002895 @default.
- W4295957051 creator A5040892104 @default.
- W4295957051 creator A5056282016 @default.
- W4295957051 creator A5057375078 @default.
- W4295957051 creator A5091908459 @default.
- W4295957051 date "2020-11-20" @default.
- W4295957051 modified "2023-09-24" @default.
- W4295957051 title "Convergence Analysis of Homotopy-SGD for non-convex optimization" @default.
- W4295957051 doi "https://doi.org/10.48550/arxiv.2011.10298" @default.
- W4295957051 hasPublicationYear "2020" @default.
- W4295957051 type Work @default.
- W4295957051 citedByCount "0" @default.
- W4295957051 crossrefType "posted-content" @default.
- W4295957051 hasAuthorship W4295957051A5031002895 @default.
- W4295957051 hasAuthorship W4295957051A5040892104 @default.
- W4295957051 hasAuthorship W4295957051A5056282016 @default.
- W4295957051 hasAuthorship W4295957051A5057375078 @default.
- W4295957051 hasAuthorship W4295957051A5091908459 @default.
- W4295957051 hasBestOaLocation W42959570511 @default.
- W4295957051 hasConcept C11413529 @default.
- W4295957051 hasConcept C117160843 @default.
- W4295957051 hasConcept C118615104 @default.
- W4295957051 hasConcept C126255220 @default.
- W4295957051 hasConcept C127162648 @default.
- W4295957051 hasConcept C127705205 @default.
- W4295957051 hasConcept C134306372 @default.
- W4295957051 hasConcept C140479938 @default.
- W4295957051 hasConcept C154945302 @default.
- W4295957051 hasConcept C162324750 @default.
- W4295957051 hasConcept C194387892 @default.
- W4295957051 hasConcept C202444582 @default.
- W4295957051 hasConcept C206688291 @default.
- W4295957051 hasConcept C2777303404 @default.
- W4295957051 hasConcept C28826006 @default.
- W4295957051 hasConcept C31258907 @default.
- W4295957051 hasConcept C33923547 @default.
- W4295957051 hasConcept C41008148 @default.
- W4295957051 hasConcept C50522688 @default.
- W4295957051 hasConcept C50644808 @default.
- W4295957051 hasConcept C57869625 @default.
- W4295957051 hasConcept C5961521 @default.
- W4295957051 hasConceptScore W4295957051C11413529 @default.
- W4295957051 hasConceptScore W4295957051C117160843 @default.
- W4295957051 hasConceptScore W4295957051C118615104 @default.
- W4295957051 hasConceptScore W4295957051C126255220 @default.
- W4295957051 hasConceptScore W4295957051C127162648 @default.
- W4295957051 hasConceptScore W4295957051C127705205 @default.
- W4295957051 hasConceptScore W4295957051C134306372 @default.
- W4295957051 hasConceptScore W4295957051C140479938 @default.
- W4295957051 hasConceptScore W4295957051C154945302 @default.
- W4295957051 hasConceptScore W4295957051C162324750 @default.
- W4295957051 hasConceptScore W4295957051C194387892 @default.
- W4295957051 hasConceptScore W4295957051C202444582 @default.
- W4295957051 hasConceptScore W4295957051C206688291 @default.
- W4295957051 hasConceptScore W4295957051C2777303404 @default.
- W4295957051 hasConceptScore W4295957051C28826006 @default.
- W4295957051 hasConceptScore W4295957051C31258907 @default.
- W4295957051 hasConceptScore W4295957051C33923547 @default.
- W4295957051 hasConceptScore W4295957051C41008148 @default.
- W4295957051 hasConceptScore W4295957051C50522688 @default.
- W4295957051 hasConceptScore W4295957051C50644808 @default.
- W4295957051 hasConceptScore W4295957051C57869625 @default.
- W4295957051 hasConceptScore W4295957051C5961521 @default.
- W4295957051 hasLocation W42959570511 @default.
- W4295957051 hasOpenAccess W4295957051 @default.
- W4295957051 hasPrimaryLocation W42959570511 @default.
- W4295957051 hasRelatedWork W2075181955 @default.
- W4295957051 hasRelatedWork W2238261533 @default.
- W4295957051 hasRelatedWork W2398337421 @default.
- W4295957051 hasRelatedWork W2746934669 @default.
- W4295957051 hasRelatedWork W2780752111 @default.
- W4295957051 hasRelatedWork W3108574418 @default.
- W4295957051 hasRelatedWork W3152995848 @default.
- W4295957051 hasRelatedWork W4282813220 @default.
- W4295957051 hasRelatedWork W4298055110 @default.
- W4295957051 hasRelatedWork W4300993590 @default.
- W4295957051 isParatext "false" @default.
- W4295957051 isRetracted "false" @default.
- W4295957051 workType "article" @default.