Matches in SemOpenAlex for { <https://semopenalex.org/work/W2938247478> ?p ?o ?g. }
Showing items 1 to 88 of
88
with 100 items per page.
- W2938247478 abstract "Stochastic mirror descent (SMD) algorithms have recently garnered a great deal of attention in optimization, signal processing, and machine learning. They are similar to stochastic gradient descent (SGD), in that they perform updates along the negative gradient of an instantaneous (or stochastically chosen) loss function. However, rather than update the parameter (or weight) vector directly, they update it in a mirrored domain whose transformation is given by the gradient of a strictly convex differentiable potential function. SMD was originally conceived to take advantage of the underlying geometry of the problem as a way to improve the convergence rate over SGD. In this paper, we study SMD, for linear models and convex loss functions, through the lens of H <sup xmlns:mml=http://www.w3.org/1998/Math/MathML xmlns:xlink=http://www.w3.org/1999/xlink>∞</sup> estimation theory and come up with a minimax interpretation of the SMD algorithm which is the counterpart of the H <sup xmlns:mml=http://www.w3.org/1998/Math/MathML xmlns:xlink=http://www.w3.org/1999/xlink>∞</sup> -optimality of the SGD algorithm for linear models and quadratic loss. In doing so, we identify a fundamental conservation law that SMD satisfies and use it to study the convergence properties of the algorithm. For constant step size SMD, when the linear model is over-parameterized, we give a deterministic proof of convergence for SMD and show that from any initial point, it converges to the closest point in the space of all parameter vectors that interpolate the data, where closest is in the sense of the Bregman divergence of the potential function. This property is referred to as implicit regularization: with an appropriate choice of the potential function one can guarantee convergence to the minimizer of any desired convex regularizer. For vanishing step size SMD, and in the standard stochastic optimization setting, we give a direct and elementary proof of convergence for SMD to the true parameter vector which avoids ergodic averaging or appealing to stochastic differential equations." @default.
- W2938247478 created "2019-04-25" @default.
- W2938247478 creator A5002430773 @default.
- W2938247478 creator A5005748450 @default.
- W2938247478 date "2019-05-01" @default.
- W2938247478 modified "2023-09-26" @default.
- W2938247478 title "A Characterization of Stochastic Mirror Descent Algorithms and Their Convergence Properties" @default.
- W2938247478 cites W111418999 @default.
- W2938247478 cites W1540586255 @default.
- W2938247478 cites W1553032475 @default.
- W2938247478 cites W2016384870 @default.
- W2938247478 cites W2102800374 @default.
- W2938247478 cites W2156291289 @default.
- W2938247478 cites W2794413559 @default.
- W2938247478 doi "https://doi.org/10.1109/icassp.2019.8682271" @default.
- W2938247478 hasPublicationYear "2019" @default.
- W2938247478 type Work @default.
- W2938247478 sameAs 2938247478 @default.
- W2938247478 citedByCount "7" @default.
- W2938247478 countsByYear W29382474782019 @default.
- W2938247478 countsByYear W29382474782020 @default.
- W2938247478 countsByYear W29382474782023 @default.
- W2938247478 crossrefType "proceedings-article" @default.
- W2938247478 hasAuthorship W2938247478A5002430773 @default.
- W2938247478 hasAuthorship W2938247478A5005748450 @default.
- W2938247478 hasConcept C112680207 @default.
- W2938247478 hasConcept C11413529 @default.
- W2938247478 hasConcept C126255220 @default.
- W2938247478 hasConcept C127162648 @default.
- W2938247478 hasConcept C14036430 @default.
- W2938247478 hasConcept C145446738 @default.
- W2938247478 hasConcept C149728462 @default.
- W2938247478 hasConcept C153258448 @default.
- W2938247478 hasConcept C154945302 @default.
- W2938247478 hasConcept C162324750 @default.
- W2938247478 hasConcept C165464430 @default.
- W2938247478 hasConcept C206688291 @default.
- W2938247478 hasConcept C2524010 @default.
- W2938247478 hasConcept C2777303404 @default.
- W2938247478 hasConcept C28826006 @default.
- W2938247478 hasConcept C31258907 @default.
- W2938247478 hasConcept C33923547 @default.
- W2938247478 hasConcept C41008148 @default.
- W2938247478 hasConcept C50522688 @default.
- W2938247478 hasConcept C50644808 @default.
- W2938247478 hasConcept C57869625 @default.
- W2938247478 hasConcept C78458016 @default.
- W2938247478 hasConcept C86803240 @default.
- W2938247478 hasConceptScore W2938247478C112680207 @default.
- W2938247478 hasConceptScore W2938247478C11413529 @default.
- W2938247478 hasConceptScore W2938247478C126255220 @default.
- W2938247478 hasConceptScore W2938247478C127162648 @default.
- W2938247478 hasConceptScore W2938247478C14036430 @default.
- W2938247478 hasConceptScore W2938247478C145446738 @default.
- W2938247478 hasConceptScore W2938247478C149728462 @default.
- W2938247478 hasConceptScore W2938247478C153258448 @default.
- W2938247478 hasConceptScore W2938247478C154945302 @default.
- W2938247478 hasConceptScore W2938247478C162324750 @default.
- W2938247478 hasConceptScore W2938247478C165464430 @default.
- W2938247478 hasConceptScore W2938247478C206688291 @default.
- W2938247478 hasConceptScore W2938247478C2524010 @default.
- W2938247478 hasConceptScore W2938247478C2777303404 @default.
- W2938247478 hasConceptScore W2938247478C28826006 @default.
- W2938247478 hasConceptScore W2938247478C31258907 @default.
- W2938247478 hasConceptScore W2938247478C33923547 @default.
- W2938247478 hasConceptScore W2938247478C41008148 @default.
- W2938247478 hasConceptScore W2938247478C50522688 @default.
- W2938247478 hasConceptScore W2938247478C50644808 @default.
- W2938247478 hasConceptScore W2938247478C57869625 @default.
- W2938247478 hasConceptScore W2938247478C78458016 @default.
- W2938247478 hasConceptScore W2938247478C86803240 @default.
- W2938247478 hasLocation W29382474781 @default.
- W2938247478 hasOpenAccess W2938247478 @default.
- W2938247478 hasPrimaryLocation W29382474781 @default.
- W2938247478 hasRelatedWork W2587943776 @default.
- W2938247478 hasRelatedWork W2928207644 @default.
- W2938247478 hasRelatedWork W2948978298 @default.
- W2938247478 hasRelatedWork W2964336028 @default.
- W2938247478 hasRelatedWork W3057525668 @default.
- W2938247478 hasRelatedWork W3214324039 @default.
- W2938247478 hasRelatedWork W4283069798 @default.
- W2938247478 hasRelatedWork W4288376397 @default.
- W2938247478 hasRelatedWork W4296608771 @default.
- W2938247478 hasRelatedWork W4299804606 @default.
- W2938247478 isParatext "false" @default.
- W2938247478 isRetracted "false" @default.
- W2938247478 magId "2938247478" @default.
- W2938247478 workType "article" @default.