Matches in SemOpenAlex for { <https://semopenalex.org/work/W4288365839> ?p ?o ?g. }
Showing items 1 to 67 of
67
with 100 items per page.
- W4288365839 abstract "We consider networks, trained via stochastic gradient descent to minimize $ell_2$ loss, with the training labels perturbed by independent noise at each iteration. We characterize the behavior of the training dynamics near any parameter vector that achieves zero training error, in terms of an implicit regularization term corresponding to the sum over the data points, of the squared $ell_2$ norm of the gradient of the model with respect to the parameter vector, evaluated at each data point. This holds for networks of any connectivity, width, depth, and choice of activation function. We interpret this implicit regularization term for three simple settings: matrix sensing, two layer ReLU networks trained on one-dimensional data, and two layer networks with sigmoid activations trained on a single datapoint. For these settings, we show why this new and general implicit regularization effect drives the networks towards simple models." @default.
- W4288365839 created "2022-07-29" @default.
- W4288365839 creator A5036230157 @default.
- W4288365839 creator A5059516326 @default.
- W4288365839 creator A5079503799 @default.
- W4288365839 creator A5088223931 @default.
- W4288365839 date "2019-04-19" @default.
- W4288365839 modified "2023-09-30" @default.
- W4288365839 title "Implicit regularization for deep neural networks driven by an Ornstein-Uhlenbeck like process" @default.
- W4288365839 doi "https://doi.org/10.48550/arxiv.1904.09080" @default.
- W4288365839 hasPublicationYear "2019" @default.
- W4288365839 type Work @default.
- W4288365839 citedByCount "0" @default.
- W4288365839 crossrefType "posted-content" @default.
- W4288365839 hasAuthorship W4288365839A5036230157 @default.
- W4288365839 hasAuthorship W4288365839A5059516326 @default.
- W4288365839 hasAuthorship W4288365839A5079503799 @default.
- W4288365839 hasAuthorship W4288365839A5088223931 @default.
- W4288365839 hasBestOaLocation W42883658391 @default.
- W4288365839 hasConcept C111472728 @default.
- W4288365839 hasConcept C11413529 @default.
- W4288365839 hasConcept C138885662 @default.
- W4288365839 hasConcept C153258448 @default.
- W4288365839 hasConcept C154945302 @default.
- W4288365839 hasConcept C17744445 @default.
- W4288365839 hasConcept C191795146 @default.
- W4288365839 hasConcept C199539241 @default.
- W4288365839 hasConcept C206688291 @default.
- W4288365839 hasConcept C2776135515 @default.
- W4288365839 hasConcept C2780586882 @default.
- W4288365839 hasConcept C28826006 @default.
- W4288365839 hasConcept C33923547 @default.
- W4288365839 hasConcept C41008148 @default.
- W4288365839 hasConcept C50644808 @default.
- W4288365839 hasConcept C81388566 @default.
- W4288365839 hasConceptScore W4288365839C111472728 @default.
- W4288365839 hasConceptScore W4288365839C11413529 @default.
- W4288365839 hasConceptScore W4288365839C138885662 @default.
- W4288365839 hasConceptScore W4288365839C153258448 @default.
- W4288365839 hasConceptScore W4288365839C154945302 @default.
- W4288365839 hasConceptScore W4288365839C17744445 @default.
- W4288365839 hasConceptScore W4288365839C191795146 @default.
- W4288365839 hasConceptScore W4288365839C199539241 @default.
- W4288365839 hasConceptScore W4288365839C206688291 @default.
- W4288365839 hasConceptScore W4288365839C2776135515 @default.
- W4288365839 hasConceptScore W4288365839C2780586882 @default.
- W4288365839 hasConceptScore W4288365839C28826006 @default.
- W4288365839 hasConceptScore W4288365839C33923547 @default.
- W4288365839 hasConceptScore W4288365839C41008148 @default.
- W4288365839 hasConceptScore W4288365839C50644808 @default.
- W4288365839 hasConceptScore W4288365839C81388566 @default.
- W4288365839 hasLocation W42883658391 @default.
- W4288365839 hasOpenAccess W4288365839 @default.
- W4288365839 hasPrimaryLocation W42883658391 @default.
- W4288365839 hasRelatedWork W2018863220 @default.
- W4288365839 hasRelatedWork W2940107683 @default.
- W4288365839 hasRelatedWork W3022706495 @default.
- W4288365839 hasRelatedWork W3046739385 @default.
- W4288365839 hasRelatedWork W3094062549 @default.
- W4288365839 hasRelatedWork W3141318533 @default.
- W4288365839 hasRelatedWork W4206520803 @default.
- W4288365839 hasRelatedWork W4288365839 @default.
- W4288365839 hasRelatedWork W4289926321 @default.
- W4288365839 hasRelatedWork W4297780756 @default.
- W4288365839 isParatext "false" @default.
- W4288365839 isRetracted "false" @default.
- W4288365839 workType "article" @default.