Matches in SemOpenAlex for { <https://semopenalex.org/work/W3128324989> ?p ?o ?g. }
- W3128324989 abstract "In this paper, we introduce the Layer-Peeled Model, a nonconvex yet analytically tractable optimization program, in a quest to better understand deep neural networks that are trained for a sufficiently long time. As the name suggests, this new model is derived by isolating the topmost layer from the remainder of the neural network, followed by imposing certain constraints separately on the two parts. We demonstrate that the Layer-Peeled Model, albeit simple, inherits many characteristics of well-trained neural networks, thereby offering an effective tool for explaining and predicting common empirical patterns of deep learning training. First, when working on class-balanced datasets, we prove that any solution to this model forms a simplex equiangular tight frame, which in part explains the recently discovered phenomenon of neural collapse in deep learning training [PHD20]. Moreover, when moving to the imbalanced case, our analysis of the Layer-Peeled Model reveals a hitherto unknown phenomenon that we term Minority Collapse, which fundamentally limits the performance of deep learning models on the minority classes. In addition, we use the Layer-Peeled Model to gain insights into how to mitigate Minority Collapse. Interestingly, this phenomenon is first predicted by the Layer-Peeled Model before its confirmation by our computational experiments." @default.
- W3128324989 created "2021-02-15" @default.
- W3128324989 creator A5005455141 @default.
- W3128324989 creator A5008843158 @default.
- W3128324989 creator A5051237708 @default.
- W3128324989 creator A5080575294 @default.
- W3128324989 date "2021-01-29" @default.
- W3128324989 modified "2023-09-27" @default.
- W3128324989 title "Layer-Peeled Model: Toward Understanding Well-Trained Deep Neural Networks." @default.
- W3128324989 cites W1686810756 @default.
- W3128324989 cites W2086869478 @default.
- W3128324989 cites W2088911157 @default.
- W3128324989 cites W2110540077 @default.
- W3128324989 cites W2170402521 @default.
- W3128324989 cites W2194775991 @default.
- W3128324989 cites W2250539671 @default.
- W3128324989 cites W2257979135 @default.
- W3128324989 cites W2528305538 @default.
- W3128324989 cites W2551429935 @default.
- W3128324989 cites W2709553318 @default.
- W3128324989 cites W2750384547 @default.
- W3128324989 cites W2767106145 @default.
- W3128324989 cites W2809090039 @default.
- W3128324989 cites W2899748887 @default.
- W3128324989 cites W2900959181 @default.
- W3128324989 cites W2911847432 @default.
- W3128324989 cites W2938647293 @default.
- W3128324989 cites W2939937248 @default.
- W3128324989 cites W2947380870 @default.
- W3128324989 cites W2951585248 @default.
- W3128324989 cites W2952120674 @default.
- W3128324989 cites W2952271367 @default.
- W3128324989 cites W2963433607 @default.
- W3128324989 cites W2963518130 @default.
- W3128324989 cites W2970971581 @default.
- W3128324989 cites W2994927236 @default.
- W3128324989 cites W2995658364 @default.
- W3128324989 cites W3005680577 @default.
- W3128324989 cites W3016499390 @default.
- W3128324989 cites W3018378048 @default.
- W3128324989 cites W3035664806 @default.
- W3128324989 cites W3036601975 @default.
- W3128324989 cites W3048202181 @default.
- W3128324989 cites W3065974826 @default.
- W3128324989 cites W3099782249 @default.
- W3128324989 cites W3109311007 @default.
- W3128324989 cites W3111698566 @default.
- W3128324989 cites W3112867725 @default.
- W3128324989 cites W3116968400 @default.
- W3128324989 cites W3118608800 @default.
- W3128324989 cites W3119009163 @default.
- W3128324989 cites W3155726402 @default.
- W3128324989 cites W3175488619 @default.
- W3128324989 cites W3181414820 @default.
- W3128324989 hasPublicationYear "2021" @default.
- W3128324989 type Work @default.
- W3128324989 sameAs 3128324989 @default.
- W3128324989 citedByCount "3" @default.
- W3128324989 countsByYear W31283249892021 @default.
- W3128324989 crossrefType "posted-content" @default.
- W3128324989 hasAuthorship W3128324989A5005455141 @default.
- W3128324989 hasAuthorship W3128324989A5008843158 @default.
- W3128324989 hasAuthorship W3128324989A5051237708 @default.
- W3128324989 hasAuthorship W3128324989A5080575294 @default.
- W3128324989 hasConcept C108583219 @default.
- W3128324989 hasConcept C111472728 @default.
- W3128324989 hasConcept C11413529 @default.
- W3128324989 hasConcept C119857082 @default.
- W3128324989 hasConcept C121332964 @default.
- W3128324989 hasConcept C134306372 @default.
- W3128324989 hasConcept C138885662 @default.
- W3128324989 hasConcept C144521790 @default.
- W3128324989 hasConcept C151201525 @default.
- W3128324989 hasConcept C154945302 @default.
- W3128324989 hasConcept C178790620 @default.
- W3128324989 hasConcept C185592680 @default.
- W3128324989 hasConcept C2524010 @default.
- W3128324989 hasConcept C2777212361 @default.
- W3128324989 hasConcept C2779227376 @default.
- W3128324989 hasConcept C2780586882 @default.
- W3128324989 hasConcept C2984842247 @default.
- W3128324989 hasConcept C33923547 @default.
- W3128324989 hasConcept C41008148 @default.
- W3128324989 hasConcept C41045048 @default.
- W3128324989 hasConcept C50335755 @default.
- W3128324989 hasConcept C50644808 @default.
- W3128324989 hasConcept C62438384 @default.
- W3128324989 hasConcept C62520636 @default.
- W3128324989 hasConceptScore W3128324989C108583219 @default.
- W3128324989 hasConceptScore W3128324989C111472728 @default.
- W3128324989 hasConceptScore W3128324989C11413529 @default.
- W3128324989 hasConceptScore W3128324989C119857082 @default.
- W3128324989 hasConceptScore W3128324989C121332964 @default.
- W3128324989 hasConceptScore W3128324989C134306372 @default.
- W3128324989 hasConceptScore W3128324989C138885662 @default.
- W3128324989 hasConceptScore W3128324989C144521790 @default.
- W3128324989 hasConceptScore W3128324989C151201525 @default.
- W3128324989 hasConceptScore W3128324989C154945302 @default.
- W3128324989 hasConceptScore W3128324989C178790620 @default.
- W3128324989 hasConceptScore W3128324989C185592680 @default.