FRINK Linked Data Fragments server

Matches in SemOpenAlex for { <https://semopenalex.org/work/W2806970110> ?p ?o ?g. }

• W2806970110 abstract "Stochastic descent methods (of the gradient and mirror varieties) have become increasingly popular in optimization. In fact, it is now widely recognized that the success of deep learning is not only due to the special deep architecture of the models, but also due to the behavior of the stochastic descent methods used, which play a key role in reaching good solutions that generalize well to unseen data. In an attempt to shed some light on why this is the case, we revisit some minimax properties of stochastic gradient descent (SGD) for the square loss of linear models---originally developed in the 1990's---and extend them to general stochastic mirror descent (SMD) algorithms for general loss functions and nonlinear models. In particular, we show that there is a fundamental identity which holds for SMD (and SGD) under very general conditions, and which implies the minimax optimality of SMD (and SGD) for sufficiently small step size, and for a general class of loss functions and general nonlinear models. We further show that this identity can be used to naturally establish other properties of SMD (and SGD), namely convergence and implicit regularization for linear models (in what is now being called the interpolating regime), some of which have been shown in certain cases in prior literature. We also argue how this identity can be used in the so-called highly over-parameterized nonlinear setting (where the number of parameters far exceeds the number of data points) to provide insights into why SMD (and SGD) may have similar convergence and implicit regularization properties for deep learning." @default.
• W2806970110 created "2018-06-13" @default.
• W2806970110 creator A5002430773 @default.
• W2806970110 creator A5005748450 @default.
• W2806970110 date "2018-06-04" @default.
• W2806970110 modified "2023-09-27" @default.
• W2806970110 title "Stochastic Gradient/Mirror Descent: Minimax Optimality and Implicit Regularization" @default.
• W2806970110 cites W111418999 @default.
• W2806970110 cites W1505731132 @default.
• W2806970110 cites W1522301498 @default.
• W2806970110 cites W1540586255 @default.
• W2806970110 cites W1553032475 @default.
• W2806970110 cites W1579118835 @default.
• W2806970110 cites W1603631609 @default.
• W2806970110 cites W2016384870 @default.
• W2806970110 cites W2077723394 @default.
• W2806970110 cites W2102800374 @default.
• W2806970110 cites W2120240656 @default.
• W2806970110 cites W2128668667 @default.
• W2806970110 cites W2145339207 @default.
• W2806970110 cites W2146502635 @default.
• W2806970110 cites W2156291289 @default.
• W2806970110 cites W2163605009 @default.
• W2806970110 cites W2257979135 @default.
• W2806970110 cites W2474090883 @default.
• W2806970110 cites W2513180554 @default.
• W2806970110 cites W2525778437 @default.
• W2806970110 cites W2593634001 @default.
• W2806970110 cites W2613715972 @default.
• W2806970110 cites W2622255292 @default.
• W2806970110 cites W2768503813 @default.
• W2806970110 cites W2788067318 @default.
• W2806970110 cites W2805926765 @default.
• W2806970110 cites W2809994596 @default.
• W2806970110 cites W2899476926 @default.
• W2806970110 cites W2911742574 @default.
• W2806970110 cites W2919115771 @default.
• W2806970110 cites W2950220847 @default.
• W2806970110 cites W2962712496 @default.
• W2806970110 cites W2962807687 @default.
• W2806970110 cites W2963177640 @default.
• W2806970110 cites W2963208657 @default.
• W2806970110 cites W2963417959 @default.
• W2806970110 cites W2963446085 @default.
• W2806970110 cites W1846446278 @default.
• W2806970110 hasPublicationYear "2018" @default.
• W2806970110 type Work @default.
• W2806970110 sameAs 2806970110 @default.
• W2806970110 citedByCount "4" @default.
• W2806970110 countsByYear W28069701102018 @default.
• W2806970110 countsByYear W28069701102019 @default.
• W2806970110 countsByYear W28069701102020 @default.
• W2806970110 crossrefType "posted-content" @default.
• W2806970110 hasAuthorship W2806970110A5002430773 @default.
• W2806970110 hasAuthorship W2806970110A5005748450 @default.
• W2806970110 hasConcept C11413529 @default.
• W2806970110 hasConcept C121332964 @default.
• W2806970110 hasConcept C126255220 @default.
• W2806970110 hasConcept C149728462 @default.
• W2806970110 hasConcept C153258448 @default.
• W2806970110 hasConcept C154945302 @default.
• W2806970110 hasConcept C158622935 @default.
• W2806970110 hasConcept C162324750 @default.
• W2806970110 hasConcept C165464430 @default.
• W2806970110 hasConcept C194387892 @default.
• W2806970110 hasConcept C206688291 @default.
• W2806970110 hasConcept C2776135515 @default.
• W2806970110 hasConcept C2777303404 @default.
• W2806970110 hasConcept C28826006 @default.
• W2806970110 hasConcept C33923547 @default.
• W2806970110 hasConcept C41008148 @default.
• W2806970110 hasConcept C50522688 @default.
• W2806970110 hasConcept C50644808 @default.
• W2806970110 hasConcept C62520636 @default.
• W2806970110 hasConceptScore W2806970110C11413529 @default.
• W2806970110 hasConceptScore W2806970110C121332964 @default.
• W2806970110 hasConceptScore W2806970110C126255220 @default.
• W2806970110 hasConceptScore W2806970110C149728462 @default.
• W2806970110 hasConceptScore W2806970110C153258448 @default.
• W2806970110 hasConceptScore W2806970110C154945302 @default.
• W2806970110 hasConceptScore W2806970110C158622935 @default.
• W2806970110 hasConceptScore W2806970110C162324750 @default.
• W2806970110 hasConceptScore W2806970110C165464430 @default.
• W2806970110 hasConceptScore W2806970110C194387892 @default.
• W2806970110 hasConceptScore W2806970110C206688291 @default.
• W2806970110 hasConceptScore W2806970110C2776135515 @default.
• W2806970110 hasConceptScore W2806970110C2777303404 @default.
• W2806970110 hasConceptScore W2806970110C28826006 @default.
• W2806970110 hasConceptScore W2806970110C33923547 @default.
• W2806970110 hasConceptScore W2806970110C41008148 @default.
• W2806970110 hasConceptScore W2806970110C50522688 @default.
• W2806970110 hasConceptScore W2806970110C50644808 @default.
• W2806970110 hasConceptScore W2806970110C62520636 @default.
• W2806970110 hasLocation W28069701101 @default.
• W2806970110 hasOpenAccess W2806970110 @default.
• W2806970110 hasPrimaryLocation W28069701101 @default.
• W2806970110 hasRelatedWork W1513540972 @default.
• W2806970110 hasRelatedWork W2181730996 @default.
• W2806970110 hasRelatedWork W2239951326 @default.
• W2806970110 hasRelatedWork W2429903989 @default.

• next