Matches in SemOpenAlex for { <https://semopenalex.org/work/W3093969881> ?p ?o ?g. }
Showing items 1 to 87 of
87
with 100 items per page.
- W3093969881 abstract "A remarkable characteristic of overparameterized deep neural networks (DNNs) is that their accuracy does not degrade when the network's width is increased. Recent evidence suggests that developing compressible representations is key for adjusting the complexity of large networks to the learning task at hand. However, these compressible representations are poorly understood. A promising strand of research inspired from biology is understanding representations at the unit level as it offers a more granular and intuitive interpretation of the neural mechanisms. In order to better understand what facilitates increases in width without decreases in accuracy, we ask: Are there mechanisms at the unit level by which networks control their effective complexity as their width is increased? If so, how do these depend on the architecture, dataset, and training parameters? We identify two distinct types of units that proliferate when the network's width is increased: prunable units which can be dropped out of the network without significant change to the output and redundant units whose activities can be expressed as a linear combination of others. These units imply complexity constraints as the function the network represents could be expressed by a network without them. We also identify how the development of these units can be influenced by architecture and a number of training factors. Together, these results help to explain why the accuracy of DNNs does not degrade when width is increased and highlight the importance of frivolous units toward understanding implicit regularization in DNNs." @default.
- W3093969881 created "2020-10-29" @default.
- W3093969881 creator A5008237906 @default.
- W3093969881 creator A5012917685 @default.
- W3093969881 creator A5015476752 @default.
- W3093969881 creator A5020830959 @default.
- W3093969881 creator A5032245448 @default.
- W3093969881 creator A5034283312 @default.
- W3093969881 creator A5081541767 @default.
- W3093969881 date "2019-12-10" @default.
- W3093969881 modified "2023-09-27" @default.
- W3093969881 title "Frivolous Units Help to Explain Non-Overfitting in Overparametrized Deep Neural Networks" @default.
- W3093969881 hasPublicationYear "2019" @default.
- W3093969881 type Work @default.
- W3093969881 sameAs 3093969881 @default.
- W3093969881 citedByCount "0" @default.
- W3093969881 crossrefType "posted-content" @default.
- W3093969881 hasAuthorship W3093969881A5008237906 @default.
- W3093969881 hasAuthorship W3093969881A5012917685 @default.
- W3093969881 hasAuthorship W3093969881A5015476752 @default.
- W3093969881 hasAuthorship W3093969881A5020830959 @default.
- W3093969881 hasAuthorship W3093969881A5032245448 @default.
- W3093969881 hasAuthorship W3093969881A5034283312 @default.
- W3093969881 hasAuthorship W3093969881A5081541767 @default.
- W3093969881 hasConcept C119857082 @default.
- W3093969881 hasConcept C123657996 @default.
- W3093969881 hasConcept C14036430 @default.
- W3093969881 hasConcept C142362112 @default.
- W3093969881 hasConcept C153349607 @default.
- W3093969881 hasConcept C154945302 @default.
- W3093969881 hasConcept C162324750 @default.
- W3093969881 hasConcept C187736073 @default.
- W3093969881 hasConcept C193415008 @default.
- W3093969881 hasConcept C22019652 @default.
- W3093969881 hasConcept C2776135515 @default.
- W3093969881 hasConcept C2780451532 @default.
- W3093969881 hasConcept C2984842247 @default.
- W3093969881 hasConcept C38652104 @default.
- W3093969881 hasConcept C41008148 @default.
- W3093969881 hasConcept C50644808 @default.
- W3093969881 hasConcept C78458016 @default.
- W3093969881 hasConcept C86803240 @default.
- W3093969881 hasConceptScore W3093969881C119857082 @default.
- W3093969881 hasConceptScore W3093969881C123657996 @default.
- W3093969881 hasConceptScore W3093969881C14036430 @default.
- W3093969881 hasConceptScore W3093969881C142362112 @default.
- W3093969881 hasConceptScore W3093969881C153349607 @default.
- W3093969881 hasConceptScore W3093969881C154945302 @default.
- W3093969881 hasConceptScore W3093969881C162324750 @default.
- W3093969881 hasConceptScore W3093969881C187736073 @default.
- W3093969881 hasConceptScore W3093969881C193415008 @default.
- W3093969881 hasConceptScore W3093969881C22019652 @default.
- W3093969881 hasConceptScore W3093969881C2776135515 @default.
- W3093969881 hasConceptScore W3093969881C2780451532 @default.
- W3093969881 hasConceptScore W3093969881C2984842247 @default.
- W3093969881 hasConceptScore W3093969881C38652104 @default.
- W3093969881 hasConceptScore W3093969881C41008148 @default.
- W3093969881 hasConceptScore W3093969881C50644808 @default.
- W3093969881 hasConceptScore W3093969881C78458016 @default.
- W3093969881 hasConceptScore W3093969881C86803240 @default.
- W3093969881 hasLocation W30939698811 @default.
- W3093969881 hasOpenAccess W3093969881 @default.
- W3093969881 hasPrimaryLocation W30939698811 @default.
- W3093969881 hasRelatedWork W2189560425 @default.
- W3093969881 hasRelatedWork W2247148075 @default.
- W3093969881 hasRelatedWork W2586218966 @default.
- W3093969881 hasRelatedWork W2779918130 @default.
- W3093969881 hasRelatedWork W2898211994 @default.
- W3093969881 hasRelatedWork W2926417365 @default.
- W3093969881 hasRelatedWork W2939745571 @default.
- W3093969881 hasRelatedWork W2949978219 @default.
- W3093969881 hasRelatedWork W2952659618 @default.
- W3093969881 hasRelatedWork W2962749646 @default.
- W3093969881 hasRelatedWork W2963652331 @default.
- W3093969881 hasRelatedWork W2964036823 @default.
- W3093969881 hasRelatedWork W2977834236 @default.
- W3093969881 hasRelatedWork W2988331457 @default.
- W3093969881 hasRelatedWork W2992528120 @default.
- W3093969881 hasRelatedWork W3030757405 @default.
- W3093969881 hasRelatedWork W3070904253 @default.
- W3093969881 hasRelatedWork W3085093108 @default.
- W3093969881 hasRelatedWork W3138001899 @default.
- W3093969881 hasRelatedWork W3206275068 @default.
- W3093969881 isParatext "false" @default.
- W3093969881 isRetracted "false" @default.
- W3093969881 magId "3093969881" @default.
- W3093969881 workType "article" @default.