Matches in SemOpenAlex for { <https://semopenalex.org/work/W4386804576> ?p ?o ?g. }
Showing items 1 to 100 of
100
with 100 items per page.
- W4386804576 endingPage "189" @default.
- W4386804576 startingPage "173" @default.
- W4386804576 abstract "The Minimum Description Length principle (MDL) is a formalization of Occam’s razor for model selection, which states that a good model is one that can losslessly compress the data while including the cost of describing the model itself. While MDL can naturally express the behavior of certain models such as autoencoders (that inherently compress data) most representation learning techniques do not rely on such models. Instead, they learn representations by training on general or, for self-supervised learning, pretext tasks. In this paper, we propose a new formulation of the MDL principle that relies on the concept of signal and noise, which are implicitly defined by the learning task at hand. Additionally, we introduce ways to empirically measure the complexity of the learned representations by analyzing the spectra of the point Jacobians. Under certain assumptions, we show that the singular values of the point Jacobians of Neural Networks driven by the MDL principle should follow either a power law or a lognormal distribution. Finally, we conduct experiments to evaluate the behavior of the proposed measure applied to deep neural networks on different datasets, with respect to several types of noise. We observe that the experimental spectral distribution is in agreement with the spectral distribution predicted by our MDL principle, which suggests that neural networks trained with gradient descent on noisy data implicitly abide the MDL principle." @default.
- W4386804576 created "2023-09-17" @default.
- W4386804576 creator A5020817426 @default.
- W4386804576 creator A5029553022 @default.
- W4386804576 creator A5065053376 @default.
- W4386804576 creator A5067803328 @default.
- W4386804576 creator A5070349002 @default.
- W4386804576 creator A5083517347 @default.
- W4386804576 date "2023-01-01" @default.
- W4386804576 modified "2023-10-10" @default.
- W4386804576 title "Is My Neural Net Driven by the MDL Principle?" @default.
- W4386804576 cites W1479895691 @default.
- W4386804576 cites W1501627432 @default.
- W4386804576 cites W1540226724 @default.
- W4386804576 cites W1980921095 @default.
- W4386804576 cites W1984713180 @default.
- W4386804576 cites W1999228575 @default.
- W4386804576 cites W2007339694 @default.
- W4386804576 cites W2047229728 @default.
- W4386804576 cites W2054658115 @default.
- W4386804576 cites W2078458772 @default.
- W4386804576 cites W2102098892 @default.
- W4386804576 cites W2105154442 @default.
- W4386804576 cites W2106596127 @default.
- W4386804576 cites W2107338889 @default.
- W4386804576 cites W2113832384 @default.
- W4386804576 cites W2138200515 @default.
- W4386804576 cites W2296087697 @default.
- W4386804576 cites W2566079294 @default.
- W4386804576 cites W2912497910 @default.
- W4386804576 cites W2963693826 @default.
- W4386804576 cites W2964184826 @default.
- W4386804576 cites W3042609801 @default.
- W4386804576 cites W3045909935 @default.
- W4386804576 cites W3098044966 @default.
- W4386804576 doi "https://doi.org/10.1007/978-3-031-43415-0_11" @default.
- W4386804576 hasPublicationYear "2023" @default.
- W4386804576 type Work @default.
- W4386804576 citedByCount "0" @default.
- W4386804576 crossrefType "book-chapter" @default.
- W4386804576 hasAuthorship W4386804576A5020817426 @default.
- W4386804576 hasAuthorship W4386804576A5029553022 @default.
- W4386804576 hasAuthorship W4386804576A5065053376 @default.
- W4386804576 hasAuthorship W4386804576A5067803328 @default.
- W4386804576 hasAuthorship W4386804576A5070349002 @default.
- W4386804576 hasAuthorship W4386804576A5083517347 @default.
- W4386804576 hasBestOaLocation W43868045762 @default.
- W4386804576 hasConcept C105795698 @default.
- W4386804576 hasConcept C11413529 @default.
- W4386804576 hasConcept C153180895 @default.
- W4386804576 hasConcept C154945302 @default.
- W4386804576 hasConcept C17744445 @default.
- W4386804576 hasConcept C199360897 @default.
- W4386804576 hasConcept C199539241 @default.
- W4386804576 hasConcept C2524010 @default.
- W4386804576 hasConcept C2776359362 @default.
- W4386804576 hasConcept C28719098 @default.
- W4386804576 hasConcept C33923547 @default.
- W4386804576 hasConcept C41008148 @default.
- W4386804576 hasConcept C50644808 @default.
- W4386804576 hasConcept C64802038 @default.
- W4386804576 hasConcept C78469957 @default.
- W4386804576 hasConcept C87465248 @default.
- W4386804576 hasConcept C94625758 @default.
- W4386804576 hasConceptScore W4386804576C105795698 @default.
- W4386804576 hasConceptScore W4386804576C11413529 @default.
- W4386804576 hasConceptScore W4386804576C153180895 @default.
- W4386804576 hasConceptScore W4386804576C154945302 @default.
- W4386804576 hasConceptScore W4386804576C17744445 @default.
- W4386804576 hasConceptScore W4386804576C199360897 @default.
- W4386804576 hasConceptScore W4386804576C199539241 @default.
- W4386804576 hasConceptScore W4386804576C2524010 @default.
- W4386804576 hasConceptScore W4386804576C2776359362 @default.
- W4386804576 hasConceptScore W4386804576C28719098 @default.
- W4386804576 hasConceptScore W4386804576C33923547 @default.
- W4386804576 hasConceptScore W4386804576C41008148 @default.
- W4386804576 hasConceptScore W4386804576C50644808 @default.
- W4386804576 hasConceptScore W4386804576C64802038 @default.
- W4386804576 hasConceptScore W4386804576C78469957 @default.
- W4386804576 hasConceptScore W4386804576C87465248 @default.
- W4386804576 hasConceptScore W4386804576C94625758 @default.
- W4386804576 hasLocation W43868045761 @default.
- W4386804576 hasLocation W43868045762 @default.
- W4386804576 hasOpenAccess W4386804576 @default.
- W4386804576 hasPrimaryLocation W43868045761 @default.
- W4386804576 hasRelatedWork W126108114 @default.
- W4386804576 hasRelatedWork W1718449494 @default.
- W4386804576 hasRelatedWork W1731670240 @default.
- W4386804576 hasRelatedWork W1972887468 @default.
- W4386804576 hasRelatedWork W2124026090 @default.
- W4386804576 hasRelatedWork W2949846021 @default.
- W4386804576 hasRelatedWork W2951568450 @default.
- W4386804576 hasRelatedWork W349256592 @default.
- W4386804576 hasRelatedWork W40064302 @default.
- W4386804576 hasRelatedWork W4306706995 @default.
- W4386804576 isParatext "false" @default.
- W4386804576 isRetracted "false" @default.
- W4386804576 workType "book-chapter" @default.