Matches in SemOpenAlex for { <https://semopenalex.org/work/W4226516188> ?p ?o ?g. }
Showing items 1 to 78 of
78
with 100 items per page.
- W4226516188 abstract "We prove that stochastic gradient descent (SGD) finds a solution that achieves $(1-epsilon)$ classification accuracy on the entire dataset. We do so under two main assumptions: (1. Local progress) There is consistent improvement of the model accuracy over batches. (2. Models compute simple functions) The function computed by the model is simple (has low Kolmogorov complexity). Intuitively, the above means that emph{local progress} of SGD implies emph{global progress}. Assumption 2 trivially holds for underparameterized models, hence, our work gives the first convergence guarantee for general, emph{underparameterized models}. Furthermore, this is the first result which is completely emph{model agnostic} - we don't require the model to have any specific architecture or activation function, it may not even be a neural network. Our analysis makes use of the entropy compression method, which was first introduced by Moser and Tardos in the context of the Lov'asz local lemma." @default.
- W4226516188 created "2022-05-05" @default.
- W4226516188 creator A5067916743 @default.
- W4226516188 date "2021-11-09" @default.
- W4226516188 modified "2023-09-28" @default.
- W4226516188 title "SGD Through the Lens of Kolmogorov Complexity" @default.
- W4226516188 hasPublicationYear "2021" @default.
- W4226516188 type Work @default.
- W4226516188 citedByCount "0" @default.
- W4226516188 crossrefType "posted-content" @default.
- W4226516188 hasAuthorship W4226516188A5067916743 @default.
- W4226516188 hasBestOaLocation W42265161881 @default.
- W4226516188 hasConcept C106301342 @default.
- W4226516188 hasConcept C111472728 @default.
- W4226516188 hasConcept C121332964 @default.
- W4226516188 hasConcept C138885662 @default.
- W4226516188 hasConcept C14036430 @default.
- W4226516188 hasConcept C151730666 @default.
- W4226516188 hasConcept C154945302 @default.
- W4226516188 hasConcept C162324750 @default.
- W4226516188 hasConcept C18903297 @default.
- W4226516188 hasConcept C206688291 @default.
- W4226516188 hasConcept C2777303404 @default.
- W4226516188 hasConcept C2777759810 @default.
- W4226516188 hasConcept C2779343474 @default.
- W4226516188 hasConcept C2780586882 @default.
- W4226516188 hasConcept C28826006 @default.
- W4226516188 hasConcept C33923547 @default.
- W4226516188 hasConcept C41008148 @default.
- W4226516188 hasConcept C46757340 @default.
- W4226516188 hasConcept C50522688 @default.
- W4226516188 hasConcept C50644808 @default.
- W4226516188 hasConcept C62520636 @default.
- W4226516188 hasConcept C78458016 @default.
- W4226516188 hasConcept C86803240 @default.
- W4226516188 hasConceptScore W4226516188C106301342 @default.
- W4226516188 hasConceptScore W4226516188C111472728 @default.
- W4226516188 hasConceptScore W4226516188C121332964 @default.
- W4226516188 hasConceptScore W4226516188C138885662 @default.
- W4226516188 hasConceptScore W4226516188C14036430 @default.
- W4226516188 hasConceptScore W4226516188C151730666 @default.
- W4226516188 hasConceptScore W4226516188C154945302 @default.
- W4226516188 hasConceptScore W4226516188C162324750 @default.
- W4226516188 hasConceptScore W4226516188C18903297 @default.
- W4226516188 hasConceptScore W4226516188C206688291 @default.
- W4226516188 hasConceptScore W4226516188C2777303404 @default.
- W4226516188 hasConceptScore W4226516188C2777759810 @default.
- W4226516188 hasConceptScore W4226516188C2779343474 @default.
- W4226516188 hasConceptScore W4226516188C2780586882 @default.
- W4226516188 hasConceptScore W4226516188C28826006 @default.
- W4226516188 hasConceptScore W4226516188C33923547 @default.
- W4226516188 hasConceptScore W4226516188C41008148 @default.
- W4226516188 hasConceptScore W4226516188C46757340 @default.
- W4226516188 hasConceptScore W4226516188C50522688 @default.
- W4226516188 hasConceptScore W4226516188C50644808 @default.
- W4226516188 hasConceptScore W4226516188C62520636 @default.
- W4226516188 hasConceptScore W4226516188C78458016 @default.
- W4226516188 hasConceptScore W4226516188C86803240 @default.
- W4226516188 hasLocation W42265161881 @default.
- W4226516188 hasOpenAccess W4226516188 @default.
- W4226516188 hasPrimaryLocation W42265161881 @default.
- W4226516188 hasRelatedWork W10036223 @default.
- W4226516188 hasRelatedWork W1125449 @default.
- W4226516188 hasRelatedWork W13970711 @default.
- W4226516188 hasRelatedWork W143479 @default.
- W4226516188 hasRelatedWork W1493653 @default.
- W4226516188 hasRelatedWork W15382557 @default.
- W4226516188 hasRelatedWork W222915 @default.
- W4226516188 hasRelatedWork W304693 @default.
- W4226516188 hasRelatedWork W4617096 @default.
- W4226516188 hasRelatedWork W6336603 @default.
- W4226516188 hasRelatedWork W638455 @default.
- W4226516188 hasRelatedWork W9283559 @default.
- W4226516188 hasRelatedWork W9554121 @default.
- W4226516188 hasRelatedWork W39413 @default.
- W4226516188 isParatext "false" @default.
- W4226516188 isRetracted "false" @default.
- W4226516188 workType "article" @default.