Matches in SemOpenAlex for { <https://semopenalex.org/work/W3169665235> ?p ?o ?g. }
Showing items 1 to 99 of
99
with 100 items per page.
- W3169665235 endingPage "51" @default.
- W3169665235 startingPage "1" @default.
- W3169665235 abstract "Stochastic gradient descent (SGD) is a popular algorithm for optimization problems arising in high-dimensional inference tasks. Here one produces an estimator of an unknown parameter from independent samples of data by iteratively optimizing a loss function. This loss function is random and often non-convex. We study the performance of the simplest version of SGD, namely online SGD, from a random start in the setting where the parameter space is high-dimensional. We develop nearly sharp thresholds for the number of samples needed for consistent estimation as one varies the dimension. Our thresholds depend only on an intrinsic property of the population loss which we call the information exponent. In particular, our results do not assume uniform control on the loss itself, such as convexity or uniform derivative bounds. The thresholds we obtain are polynomial in the dimension and the precise exponent depends explicitly on the information exponent. As a consequence of our results, we find that except for the simplest tasks, almost all of the data is used simply in the initial search phase to obtain non-trivial correlation with the ground truth. Upon attaining non-trivial correlation, the descent is rapid and exhibits law of large numbers type behavior. We illustrate our approach by applying it to a wide set of inference tasks such as phase retrieval, and parameter estimation for generalized linear models, online PCA, and spiked tensor models, as well as to supervised learning for single-layer networks with general activation functions." @default.
- W3169665235 created "2021-06-22" @default.
- W3169665235 creator A5039747459 @default.
- W3169665235 creator A5051194866 @default.
- W3169665235 creator A5085479654 @default.
- W3169665235 date "2021-01-01" @default.
- W3169665235 modified "2023-09-23" @default.
- W3169665235 title "Online stochastic gradient descent on non-convex losses from high-dimensional inference" @default.
- W3169665235 hasPublicationYear "2021" @default.
- W3169665235 type Work @default.
- W3169665235 sameAs 3169665235 @default.
- W3169665235 citedByCount "0" @default.
- W3169665235 crossrefType "journal-article" @default.
- W3169665235 hasAuthorship W3169665235A5039747459 @default.
- W3169665235 hasAuthorship W3169665235A5051194866 @default.
- W3169665235 hasAuthorship W3169665235A5085479654 @default.
- W3169665235 hasConcept C105795698 @default.
- W3169665235 hasConcept C106159729 @default.
- W3169665235 hasConcept C112680207 @default.
- W3169665235 hasConcept C11413529 @default.
- W3169665235 hasConcept C114614502 @default.
- W3169665235 hasConcept C126255220 @default.
- W3169665235 hasConcept C138885662 @default.
- W3169665235 hasConcept C14036430 @default.
- W3169665235 hasConcept C153258448 @default.
- W3169665235 hasConcept C154945302 @default.
- W3169665235 hasConcept C157972887 @default.
- W3169665235 hasConcept C162324750 @default.
- W3169665235 hasConcept C185429906 @default.
- W3169665235 hasConcept C206688291 @default.
- W3169665235 hasConcept C2524010 @default.
- W3169665235 hasConcept C2776214188 @default.
- W3169665235 hasConcept C2780388253 @default.
- W3169665235 hasConcept C28826006 @default.
- W3169665235 hasConcept C33676613 @default.
- W3169665235 hasConcept C33923547 @default.
- W3169665235 hasConcept C41008148 @default.
- W3169665235 hasConcept C41895202 @default.
- W3169665235 hasConcept C50644808 @default.
- W3169665235 hasConcept C72134830 @default.
- W3169665235 hasConcept C78458016 @default.
- W3169665235 hasConcept C86803240 @default.
- W3169665235 hasConceptScore W3169665235C105795698 @default.
- W3169665235 hasConceptScore W3169665235C106159729 @default.
- W3169665235 hasConceptScore W3169665235C112680207 @default.
- W3169665235 hasConceptScore W3169665235C11413529 @default.
- W3169665235 hasConceptScore W3169665235C114614502 @default.
- W3169665235 hasConceptScore W3169665235C126255220 @default.
- W3169665235 hasConceptScore W3169665235C138885662 @default.
- W3169665235 hasConceptScore W3169665235C14036430 @default.
- W3169665235 hasConceptScore W3169665235C153258448 @default.
- W3169665235 hasConceptScore W3169665235C154945302 @default.
- W3169665235 hasConceptScore W3169665235C157972887 @default.
- W3169665235 hasConceptScore W3169665235C162324750 @default.
- W3169665235 hasConceptScore W3169665235C185429906 @default.
- W3169665235 hasConceptScore W3169665235C206688291 @default.
- W3169665235 hasConceptScore W3169665235C2524010 @default.
- W3169665235 hasConceptScore W3169665235C2776214188 @default.
- W3169665235 hasConceptScore W3169665235C2780388253 @default.
- W3169665235 hasConceptScore W3169665235C28826006 @default.
- W3169665235 hasConceptScore W3169665235C33676613 @default.
- W3169665235 hasConceptScore W3169665235C33923547 @default.
- W3169665235 hasConceptScore W3169665235C41008148 @default.
- W3169665235 hasConceptScore W3169665235C41895202 @default.
- W3169665235 hasConceptScore W3169665235C50644808 @default.
- W3169665235 hasConceptScore W3169665235C72134830 @default.
- W3169665235 hasConceptScore W3169665235C78458016 @default.
- W3169665235 hasConceptScore W3169665235C86803240 @default.
- W3169665235 hasIssue "106" @default.
- W3169665235 hasLocation W31696652351 @default.
- W3169665235 hasOpenAccess W3169665235 @default.
- W3169665235 hasPrimaryLocation W31696652351 @default.
- W3169665235 hasRelatedWork W1575590565 @default.
- W3169665235 hasRelatedWork W2336674972 @default.
- W3169665235 hasRelatedWork W2467398807 @default.
- W3169665235 hasRelatedWork W2590516361 @default.
- W3169665235 hasRelatedWork W2794499538 @default.
- W3169665235 hasRelatedWork W2945773793 @default.
- W3169665235 hasRelatedWork W2951127603 @default.
- W3169665235 hasRelatedWork W2963089591 @default.
- W3169665235 hasRelatedWork W2963924474 @default.
- W3169665235 hasRelatedWork W2978995744 @default.
- W3169665235 hasRelatedWork W2982046302 @default.
- W3169665235 hasRelatedWork W3035774392 @default.
- W3169665235 hasRelatedWork W3086729494 @default.
- W3169665235 hasRelatedWork W3103789823 @default.
- W3169665235 hasRelatedWork W3116523102 @default.
- W3169665235 hasRelatedWork W3143274903 @default.
- W3169665235 hasRelatedWork W3166400281 @default.
- W3169665235 hasRelatedWork W3173723779 @default.
- W3169665235 hasRelatedWork W3201782785 @default.
- W3169665235 hasRelatedWork W3207668489 @default.
- W3169665235 hasVolume "22" @default.
- W3169665235 isParatext "false" @default.
- W3169665235 isRetracted "false" @default.
- W3169665235 magId "3169665235" @default.
- W3169665235 workType "article" @default.