Matches in SemOpenAlex for { <https://semopenalex.org/work/W2572009792> ?p ?o ?g. }
Showing items 1 to 97 of
97
with 100 items per page.
- W2572009792 abstract "How much can pruning algorithms teach us about the fundamentals of learning representations in neural networks? And how much can these fundamentals help while devising new pruning techniques? A lot, it turns out. Neural network pruning has become a topic of great interest in recent years, and many different techniques have been proposed to address this problem. The decision of what to prune and when to prune necessarily forces us to confront our assumptions about how neural networks actually learn to represent patterns in data. In this work, we set out to test several long-held hypotheses about neural network learning representations, approaches to pruning and the relevance of one in the context of the other. To accomplish this, we argue in favor of pruning whole neurons as opposed to the traditional method of pruning weights from optimally trained networks. We first review the historical literature, point out some common assumptions it makes, and propose methods to demonstrate the inherent flaws in these assumptions. We then propose our novel approach to pruning and set about analyzing the quality of the decisions it makes. Our analysis led us to question the validity of many widely-held assumptions behind pruning algorithms and the trade-offs we often make in the interest of reducing computational complexity. We discovered that there is a straightforward way, however expensive, to serially prune 40-70% of the neurons in a trained network with minimal effect on the learning representation and without any re-training. It is to be noted here that the motivation behind this work is not to propose an algorithm that would outperform all existing methods, but to shed light on what some inherent flaws in these methods can teach us about learning representations and how this can lead us to superior pruning techniques." @default.
- W2572009792 created "2017-01-26" @default.
- W2572009792 creator A5011191228 @default.
- W2572009792 creator A5016771874 @default.
- W2572009792 creator A5041711735 @default.
- W2572009792 creator A5090297119 @default.
- W2572009792 date "2016-11-04" @default.
- W2572009792 modified "2023-09-25" @default.
- W2572009792 title "The Incredible Shrinking Neural Network: New Perspectives on Learning Representations Through The Lens of Pruning" @default.
- W2572009792 cites W1561921554 @default.
- W2572009792 cites W1870611936 @default.
- W2572009792 cites W2007329174 @default.
- W2572009792 cites W2013825190 @default.
- W2572009792 cites W2095705004 @default.
- W2572009792 cites W2114766824 @default.
- W2572009792 cites W2125389748 @default.
- W2572009792 cites W2131097266 @default.
- W2572009792 cites W2134273960 @default.
- W2572009792 cites W2145085734 @default.
- W2572009792 cites W2155428879 @default.
- W2572009792 cites W2165758113 @default.
- W2572009792 cites W2952533036 @default.
- W2572009792 cites W2964299589 @default.
- W2572009792 hasPublicationYear "2016" @default.
- W2572009792 type Work @default.
- W2572009792 sameAs 2572009792 @default.
- W2572009792 citedByCount "9" @default.
- W2572009792 countsByYear W25720097922017 @default.
- W2572009792 countsByYear W25720097922018 @default.
- W2572009792 countsByYear W25720097922019 @default.
- W2572009792 countsByYear W25720097922021 @default.
- W2572009792 crossrefType "posted-content" @default.
- W2572009792 hasAuthorship W2572009792A5011191228 @default.
- W2572009792 hasAuthorship W2572009792A5016771874 @default.
- W2572009792 hasAuthorship W2572009792A5041711735 @default.
- W2572009792 hasAuthorship W2572009792A5090297119 @default.
- W2572009792 hasConcept C108010975 @default.
- W2572009792 hasConcept C108583219 @default.
- W2572009792 hasConcept C119857082 @default.
- W2572009792 hasConcept C151730666 @default.
- W2572009792 hasConcept C154945302 @default.
- W2572009792 hasConcept C158154518 @default.
- W2572009792 hasConcept C177264268 @default.
- W2572009792 hasConcept C17744445 @default.
- W2572009792 hasConcept C199360897 @default.
- W2572009792 hasConcept C199539241 @default.
- W2572009792 hasConcept C2776359362 @default.
- W2572009792 hasConcept C2779343474 @default.
- W2572009792 hasConcept C41008148 @default.
- W2572009792 hasConcept C50644808 @default.
- W2572009792 hasConcept C6557445 @default.
- W2572009792 hasConcept C86803240 @default.
- W2572009792 hasConcept C94625758 @default.
- W2572009792 hasConceptScore W2572009792C108010975 @default.
- W2572009792 hasConceptScore W2572009792C108583219 @default.
- W2572009792 hasConceptScore W2572009792C119857082 @default.
- W2572009792 hasConceptScore W2572009792C151730666 @default.
- W2572009792 hasConceptScore W2572009792C154945302 @default.
- W2572009792 hasConceptScore W2572009792C158154518 @default.
- W2572009792 hasConceptScore W2572009792C177264268 @default.
- W2572009792 hasConceptScore W2572009792C17744445 @default.
- W2572009792 hasConceptScore W2572009792C199360897 @default.
- W2572009792 hasConceptScore W2572009792C199539241 @default.
- W2572009792 hasConceptScore W2572009792C2776359362 @default.
- W2572009792 hasConceptScore W2572009792C2779343474 @default.
- W2572009792 hasConceptScore W2572009792C41008148 @default.
- W2572009792 hasConceptScore W2572009792C50644808 @default.
- W2572009792 hasConceptScore W2572009792C6557445 @default.
- W2572009792 hasConceptScore W2572009792C86803240 @default.
- W2572009792 hasConceptScore W2572009792C94625758 @default.
- W2572009792 hasLocation W25720097921 @default.
- W2572009792 hasOpenAccess W2572009792 @default.
- W2572009792 hasPrimaryLocation W25720097921 @default.
- W2572009792 hasRelatedWork W1686810756 @default.
- W2572009792 hasRelatedWork W1968607146 @default.
- W2572009792 hasRelatedWork W2114766824 @default.
- W2572009792 hasRelatedWork W2125389748 @default.
- W2572009792 hasRelatedWork W2194775991 @default.
- W2572009792 hasRelatedWork W2495425901 @default.
- W2572009792 hasRelatedWork W2515385951 @default.
- W2572009792 hasRelatedWork W2520760693 @default.
- W2572009792 hasRelatedWork W2707890836 @default.
- W2572009792 hasRelatedWork W2901632766 @default.
- W2572009792 hasRelatedWork W2921582471 @default.
- W2572009792 hasRelatedWork W2945551948 @default.
- W2572009792 hasRelatedWork W2963674932 @default.
- W2572009792 hasRelatedWork W2964299589 @default.
- W2572009792 hasRelatedWork W2964345707 @default.
- W2572009792 hasRelatedWork W2993208870 @default.
- W2572009792 hasRelatedWork W3034519970 @default.
- W2572009792 hasRelatedWork W3098205474 @default.
- W2572009792 hasRelatedWork W3134389780 @default.
- W2572009792 hasRelatedWork W3138862539 @default.
- W2572009792 isParatext "false" @default.
- W2572009792 isRetracted "false" @default.
- W2572009792 magId "2572009792" @default.
- W2572009792 workType "article" @default.