FRINK Linked Data Fragments server

Matches in SemOpenAlex for { <https://semopenalex.org/work/W3162340249> ?p ?o ?g. }

• W3162340249 abstract "Several recent works [40, 24] observed an interesting phenomenon in neural network pruning: A larger finetuning learning rate can improve the final performance significantly. Unfortunately, the reason behind it remains elusive up to date. This paper is meant to explain it through the lens of dynamical isometry [42]. Specifically, we examine neural network pruning from an unusual perspective: pruning as initialization for finetuning, and ask whether the inherited weights serve as a good initialization for the finetuning? The insights from dynamical isometry suggest a negative answer. Despite its critical role, this issue has not been well-recognized by the community so far. In this paper, we will show the understanding of this problem is very important -- on top of explaining the aforementioned mystery about the larger finetuning rate, it also unveils the mystery about the value of pruning [5, 30]. Besides a clearer theoretical understanding of pruning, resolving the problem can also bring us considerable performance benefits in practice." @default.
• W3162340249 created "2021-05-24" @default.
• W3162340249 creator A5005819096 @default.
• W3162340249 creator A5009988205 @default.
• W3162340249 creator A5021042598 @default.
• W3162340249 creator A5036001582 @default.
• W3162340249 date "2021-05-12" @default.
• W3162340249 modified "2023-09-27" @default.
• W3162340249 title "Dynamical Isometry: The Missing Ingredient for Neural Network Pruning." @default.
• W3162340249 cites W104184427 @default.
• W3162340249 cites W114517082 @default.
• W3162340249 cites W1533861849 @default.
• W3162340249 cites W1576347883 @default.
• W3162340249 cites W1665214252 @default.
• W3162340249 cites W1677182931 @default.
• W3162340249 cites W1836465849 @default.
• W3162340249 cites W1994616650 @default.
• W3162340249 cites W2097533491 @default.
• W3162340249 cites W2108598243 @default.
• W3162340249 cites W2114766824 @default.
• W3162340249 cites W2125389748 @default.
• W3162340249 cites W2134273960 @default.
• W3162340249 cites W2145085734 @default.
• W3162340249 cites W2194775991 @default.
• W3162340249 cites W2287334441 @default.
• W3162340249 cites W2604319603 @default.
• W3162340249 cites W2736953746 @default.
• W3162340249 cites W2783538964 @default.
• W3162340249 cites W2783873922 @default.
• W3162340249 cites W2788715907 @default.
• W3162340249 cites W2896409484 @default.
• W3162340249 cites W2897110360 @default.
• W3162340249 cites W2915589364 @default.
• W3162340249 cites W2945176031 @default.
• W3162340249 cites W2948879487 @default.
• W3162340249 cites W2962851801 @default.
• W3162340249 cites W2962965870 @default.
• W3162340249 cites W2963000224 @default.
• W3162340249 cites W2963037478 @default.
• W3162340249 cites W2963247446 @default.
• W3162340249 cites W2963263347 @default.
• W3162340249 cites W2963363373 @default.
• W3162340249 cites W2963382930 @default.
• W3162340249 cites W2963504252 @default.
• W3162340249 cites W2963674932 @default.
• W3162340249 cites W2963813662 @default.
• W3162340249 cites W2964065616 @default.
• W3162340249 cites W2964233199 @default.
• W3162340249 cites W2964299589 @default.
• W3162340249 cites W2964303560 @default.
• W3162340249 cites W2978081181 @default.
• W3162340249 cites W2995492258 @default.
• W3162340249 cites W2995816250 @default.
• W3162340249 cites W2996577930 @default.
• W3162340249 cites W3012561096 @default.
• W3162340249 cites W3021616083 @default.
• W3162340249 cites W3034234149 @default.
• W3162340249 cites W3035081900 @default.
• W3162340249 cites W3038041907 @default.
• W3162340249 cites W3120244898 @default.
• W3162340249 cites W3123262333 @default.
• W3162340249 cites W3124987929 @default.
• W3162340249 cites W3134389780 @default.
• W3162340249 cites W3143293593 @default.
• W3162340249 hasPublicationYear "2021" @default.
• W3162340249 type Work @default.
• W3162340249 sameAs 3162340249 @default.
• W3162340249 citedByCount "1" @default.
• W3162340249 countsByYear W31623402492021 @default.
• W3162340249 crossrefType "posted-content" @default.
• W3162340249 hasAuthorship W3162340249A5005819096 @default.
• W3162340249 hasAuthorship W3162340249A5009988205 @default.
• W3162340249 hasAuthorship W3162340249A5021042598 @default.
• W3162340249 hasAuthorship W3162340249A5036001582 @default.
• W3162340249 hasConcept C108010975 @default.
• W3162340249 hasConcept C111472728 @default.
• W3162340249 hasConcept C114466953 @default.
• W3162340249 hasConcept C119857082 @default.
• W3162340249 hasConcept C12713177 @default.
• W3162340249 hasConcept C138885662 @default.
• W3162340249 hasConcept C154945302 @default.
• W3162340249 hasConcept C199360897 @default.
• W3162340249 hasConcept C41008148 @default.
• W3162340249 hasConcept C50335755 @default.
• W3162340249 hasConcept C50644808 @default.
• W3162340249 hasConcept C59822182 @default.
• W3162340249 hasConcept C86803240 @default.
• W3162340249 hasConceptScore W3162340249C108010975 @default.
• W3162340249 hasConceptScore W3162340249C111472728 @default.
• W3162340249 hasConceptScore W3162340249C114466953 @default.
• W3162340249 hasConceptScore W3162340249C119857082 @default.
• W3162340249 hasConceptScore W3162340249C12713177 @default.
• W3162340249 hasConceptScore W3162340249C138885662 @default.
• W3162340249 hasConceptScore W3162340249C154945302 @default.
• W3162340249 hasConceptScore W3162340249C199360897 @default.
• W3162340249 hasConceptScore W3162340249C41008148 @default.
• W3162340249 hasConceptScore W3162340249C50335755 @default.
• W3162340249 hasConceptScore W3162340249C50644808 @default.
• W3162340249 hasConceptScore W3162340249C59822182 @default.
• W3162340249 hasConceptScore W3162340249C86803240 @default.

• next