Matches in SemOpenAlex for { <https://semopenalex.org/work/W4306295004> ?p ?o ?g. }
Showing items 1 to 78 of
78
with 100 items per page.
- W4306295004 abstract "We prove closed-form equations for the exact high-dimensional asymptotics of a family of first order gradient-based methods, learning an estimator (e.g. M-estimator, shallow neural network, ...) from observations on Gaussian data with empirical risk minimization. This includes widely used algorithms such as stochastic gradient descent (SGD) or Nesterov acceleration. The obtained equations match those resulting from the discretization of dynamical mean-field theory (DMFT) equations from statistical physics when applied to gradient flow. Our proof method allows us to give an explicit description of how memory kernels build up in the effective dynamics, and to include non-separable update functions, allowing datasets with non-identity covariance matrices. Finally, we provide numerical implementations of the equations for SGD with generic extensive batch-size and with constant learning rates." @default.
- W4306295004 created "2022-10-15" @default.
- W4306295004 creator A5044679630 @default.
- W4306295004 creator A5054525547 @default.
- W4306295004 creator A5068236230 @default.
- W4306295004 creator A5076357441 @default.
- W4306295004 creator A5089268172 @default.
- W4306295004 date "2022-10-12" @default.
- W4306295004 modified "2023-10-17" @default.
- W4306295004 title "Rigorous dynamical mean field theory for stochastic gradient descent methods" @default.
- W4306295004 doi "https://doi.org/10.48550/arxiv.2210.06591" @default.
- W4306295004 hasPublicationYear "2022" @default.
- W4306295004 type Work @default.
- W4306295004 citedByCount "0" @default.
- W4306295004 crossrefType "posted-content" @default.
- W4306295004 hasAuthorship W4306295004A5044679630 @default.
- W4306295004 hasAuthorship W4306295004A5054525547 @default.
- W4306295004 hasAuthorship W4306295004A5068236230 @default.
- W4306295004 hasAuthorship W4306295004A5076357441 @default.
- W4306295004 hasAuthorship W4306295004A5089268172 @default.
- W4306295004 hasBestOaLocation W43062950041 @default.
- W4306295004 hasConcept C105795698 @default.
- W4306295004 hasConcept C117896860 @default.
- W4306295004 hasConcept C121332964 @default.
- W4306295004 hasConcept C126255220 @default.
- W4306295004 hasConcept C134306372 @default.
- W4306295004 hasConcept C153258448 @default.
- W4306295004 hasConcept C154945302 @default.
- W4306295004 hasConcept C163716315 @default.
- W4306295004 hasConcept C167879884 @default.
- W4306295004 hasConcept C178650346 @default.
- W4306295004 hasConcept C185429906 @default.
- W4306295004 hasConcept C206688291 @default.
- W4306295004 hasConcept C28826006 @default.
- W4306295004 hasConcept C33923547 @default.
- W4306295004 hasConcept C41008148 @default.
- W4306295004 hasConcept C50644808 @default.
- W4306295004 hasConcept C62520636 @default.
- W4306295004 hasConcept C73000952 @default.
- W4306295004 hasConcept C74650414 @default.
- W4306295004 hasConcept C79379906 @default.
- W4306295004 hasConceptScore W4306295004C105795698 @default.
- W4306295004 hasConceptScore W4306295004C117896860 @default.
- W4306295004 hasConceptScore W4306295004C121332964 @default.
- W4306295004 hasConceptScore W4306295004C126255220 @default.
- W4306295004 hasConceptScore W4306295004C134306372 @default.
- W4306295004 hasConceptScore W4306295004C153258448 @default.
- W4306295004 hasConceptScore W4306295004C154945302 @default.
- W4306295004 hasConceptScore W4306295004C163716315 @default.
- W4306295004 hasConceptScore W4306295004C167879884 @default.
- W4306295004 hasConceptScore W4306295004C178650346 @default.
- W4306295004 hasConceptScore W4306295004C185429906 @default.
- W4306295004 hasConceptScore W4306295004C206688291 @default.
- W4306295004 hasConceptScore W4306295004C28826006 @default.
- W4306295004 hasConceptScore W4306295004C33923547 @default.
- W4306295004 hasConceptScore W4306295004C41008148 @default.
- W4306295004 hasConceptScore W4306295004C50644808 @default.
- W4306295004 hasConceptScore W4306295004C62520636 @default.
- W4306295004 hasConceptScore W4306295004C73000952 @default.
- W4306295004 hasConceptScore W4306295004C74650414 @default.
- W4306295004 hasConceptScore W4306295004C79379906 @default.
- W4306295004 hasLocation W43062950041 @default.
- W4306295004 hasLocation W43062950042 @default.
- W4306295004 hasOpenAccess W4306295004 @default.
- W4306295004 hasPrimaryLocation W43062950041 @default.
- W4306295004 hasRelatedWork W1966593406 @default.
- W4306295004 hasRelatedWork W2041896318 @default.
- W4306295004 hasRelatedWork W2081330222 @default.
- W4306295004 hasRelatedWork W2111453803 @default.
- W4306295004 hasRelatedWork W2381246980 @default.
- W4306295004 hasRelatedWork W305874435 @default.
- W4306295004 hasRelatedWork W3169190295 @default.
- W4306295004 hasRelatedWork W3213620266 @default.
- W4306295004 hasRelatedWork W4224998919 @default.
- W4306295004 hasRelatedWork W4287130391 @default.
- W4306295004 isParatext "false" @default.
- W4306295004 isRetracted "false" @default.
- W4306295004 workType "article" @default.