Matches in SemOpenAlex for { <https://semopenalex.org/work/W4386148424> ?p ?o ?g. }
Showing items 1 to 59 of
59
with 100 items per page.
- W4386148424 abstract "Deep neural networks (DNN) are singular statistical models which exhibit complex degeneracies. In this work, we illustrate how a quantity known as the emph{learning coefficient} introduced in singular learning theory quantifies precisely the degree of degeneracy in deep neural networks. Importantly, we will demonstrate that degeneracy in DNN cannot be accounted for by simply counting the number of flat directions. We propose a computationally scalable approximation of a localized version of the learning coefficient using stochastic gradient Langevin dynamics. To validate our approach, we demonstrate its accuracy in low-dimensional models with known theoretical values. Importantly, the local learning coefficient can correctly recover the ordering of degeneracy between various parameter regions of interest. An experiment on MNIST shows the local learning coefficient can reveal the inductive bias of stochastic opitmizers for more or less degenerate critical points." @default.
- W4386148424 created "2023-08-25" @default.
- W4386148424 creator A5046656905 @default.
- W4386148424 creator A5069744588 @default.
- W4386148424 creator A5082251796 @default.
- W4386148424 date "2023-08-23" @default.
- W4386148424 modified "2023-10-16" @default.
- W4386148424 title "Quantifying degeneracy in singular models via the learning coefficient" @default.
- W4386148424 doi "https://doi.org/10.48550/arxiv.2308.12108" @default.
- W4386148424 hasPublicationYear "2023" @default.
- W4386148424 type Work @default.
- W4386148424 citedByCount "0" @default.
- W4386148424 crossrefType "posted-content" @default.
- W4386148424 hasAuthorship W4386148424A5046656905 @default.
- W4386148424 hasAuthorship W4386148424A5069744588 @default.
- W4386148424 hasAuthorship W4386148424A5082251796 @default.
- W4386148424 hasBestOaLocation W43861484241 @default.
- W4386148424 hasConcept C121332964 @default.
- W4386148424 hasConcept C121864883 @default.
- W4386148424 hasConcept C154945302 @default.
- W4386148424 hasConcept C190502265 @default.
- W4386148424 hasConcept C2777727622 @default.
- W4386148424 hasConcept C28826006 @default.
- W4386148424 hasConcept C33923547 @default.
- W4386148424 hasConcept C41008148 @default.
- W4386148424 hasConcept C50644808 @default.
- W4386148424 hasConcept C60644358 @default.
- W4386148424 hasConcept C62520636 @default.
- W4386148424 hasConcept C72319582 @default.
- W4386148424 hasConcept C86803240 @default.
- W4386148424 hasConceptScore W4386148424C121332964 @default.
- W4386148424 hasConceptScore W4386148424C121864883 @default.
- W4386148424 hasConceptScore W4386148424C154945302 @default.
- W4386148424 hasConceptScore W4386148424C190502265 @default.
- W4386148424 hasConceptScore W4386148424C2777727622 @default.
- W4386148424 hasConceptScore W4386148424C28826006 @default.
- W4386148424 hasConceptScore W4386148424C33923547 @default.
- W4386148424 hasConceptScore W4386148424C41008148 @default.
- W4386148424 hasConceptScore W4386148424C50644808 @default.
- W4386148424 hasConceptScore W4386148424C60644358 @default.
- W4386148424 hasConceptScore W4386148424C62520636 @default.
- W4386148424 hasConceptScore W4386148424C72319582 @default.
- W4386148424 hasConceptScore W4386148424C86803240 @default.
- W4386148424 hasLocation W43861484241 @default.
- W4386148424 hasOpenAccess W4386148424 @default.
- W4386148424 hasPrimaryLocation W43861484241 @default.
- W4386148424 hasRelatedWork W1552514155 @default.
- W4386148424 hasRelatedWork W168460013 @default.
- W4386148424 hasRelatedWork W2064109252 @default.
- W4386148424 hasRelatedWork W2124225859 @default.
- W4386148424 hasRelatedWork W2125277162 @default.
- W4386148424 hasRelatedWork W2215112080 @default.
- W4386148424 hasRelatedWork W2955288804 @default.
- W4386148424 hasRelatedWork W3037733829 @default.
- W4386148424 hasRelatedWork W4287687395 @default.
- W4386148424 hasRelatedWork W4300428659 @default.
- W4386148424 isParatext "false" @default.
- W4386148424 isRetracted "false" @default.
- W4386148424 workType "article" @default.