FRINK Linked Data Fragments server

Matches in SemOpenAlex for { <https://semopenalex.org/work/W3205094911> ?p ?o ?g. }

• W3205094911 abstract "Understanding the training dynamics of deep learning models is perhaps a necessary step toward demystifying the effectiveness of these models. In particular, how do data from different classes gradually become separable in their feature spaces when training neural networks using stochastic gradient descent? In this study, we model the evolution of features during deep learning training using a set of stochastic differential equations (SDEs) that each corresponds to a training sample. As a crucial ingredient in our modeling strategy, each SDE contains a drift term that reflects the impact of backpropagation at an input on the features of all samples. Our main finding uncovers a sharp phase transition phenomenon regarding the {intra-class impact: if the SDEs are locally elastic in the sense that the impact is more significant on samples from the same class as the input, the features of the training data become linearly separable, meaning vanishing training loss; otherwise, the features are not separable, regardless of how long the training time is. Moreover, in the presence of local elasticity, an analysis of our SDEs shows that the emergence of a simple geometric structure called the neural collapse of the features. Taken together, our results shed light on the decisive role of local elasticity in the training dynamics of neural networks. We corroborate our theoretical analysis with experiments on a synthesized dataset of geometric shapes and CIFAR-10." @default.
• W3205094911 created "2021-10-25" @default.
• W3205094911 creator A5053725057 @default.
• W3205094911 creator A5080575294 @default.
• W3205094911 creator A5089098888 @default.
• W3205094911 date "2021-10-11" @default.
• W3205094911 modified "2023-10-17" @default.
• W3205094911 title "Imitating Deep Learning Dynamics via Locally Elastic Stochastic Differential Equations." @default.
• W3205094911 cites W1836465849 @default.
• W3205094911 cites W2163605009 @default.
• W3205094911 cites W2168263526 @default.
• W3205094911 cites W2194775991 @default.
• W3205094911 cites W2212676342 @default.
• W3205094911 cites W2257979135 @default.
• W3205094911 cites W2742947407 @default.
• W3205094911 cites W2768267830 @default.
• W3205094911 cites W2809090039 @default.
• W3205094911 cites W2886067286 @default.
• W3205094911 cites W2898365215 @default.
• W3205094911 cites W2911742574 @default.
• W3205094911 cites W2913473169 @default.
• W3205094911 cites W2919115771 @default.
• W3205094911 cites W2940934113 @default.
• W3205094911 cites W2962698540 @default.
• W3205094911 cites W2962726660 @default.
• W3205094911 cites W2962915600 @default.
• W3205094911 cites W2962973336 @default.
• W3205094911 cites W2963086136 @default.
• W3205094911 cites W2963177640 @default.
• W3205094911 cites W2963239103 @default.
• W3205094911 cites W2963341956 @default.
• W3205094911 cites W2963376662 @default.
• W3205094911 cites W2963446085 @default.
• W3205094911 cites W2963504252 @default.
• W3205094911 cites W2963702144 @default.
• W3205094911 cites W2963711523 @default.
• W3205094911 cites W2963959597 @default.
• W3205094911 cites W2964052793 @default.
• W3205094911 cites W2964072432 @default.
• W3205094911 cites W2964098911 @default.
• W3205094911 cites W2964121744 @default.
• W3205094911 cites W2964161337 @default.
• W3205094911 cites W2970474218 @default.
• W3205094911 cites W2970540478 @default.
• W3205094911 cites W2995658364 @default.
• W3205094911 cites W2996168800 @default.
• W3205094911 cites W3016499390 @default.
• W3205094911 cites W3065974826 @default.
• W3205094911 cites W3104810215 @default.
• W3205094911 cites W3106000452 @default.
• W3205094911 cites W3118608800 @default.
• W3205094911 cites W3130298903 @default.
• W3205094911 cites W3132026801 @default.
• W3205094911 cites W3137695714 @default.
• W3205094911 cites W3167951241 @default.
• W3205094911 cites W3205278400 @default.
• W3205094911 hasPublicationYear "2021" @default.
• W3205094911 type Work @default.
• W3205094911 sameAs 3205094911 @default.
• W3205094911 citedByCount "0" @default.
• W3205094911 crossrefType "posted-content" @default.
• W3205094911 hasAuthorship W3205094911A5053725057 @default.
• W3205094911 hasAuthorship W3205094911A5080575294 @default.
• W3205094911 hasAuthorship W3205094911A5089098888 @default.
• W3205094911 hasConcept C108583219 @default.
• W3205094911 hasConcept C119857082 @default.
• W3205094911 hasConcept C121332964 @default.
• W3205094911 hasConcept C121864883 @default.
• W3205094911 hasConcept C126255220 @default.
• W3205094911 hasConcept C134306372 @default.
• W3205094911 hasConcept C138885662 @default.
• W3205094911 hasConcept C154945302 @default.
• W3205094911 hasConcept C155032097 @default.
• W3205094911 hasConcept C206688291 @default.
• W3205094911 hasConcept C2776401178 @default.
• W3205094911 hasConcept C28826006 @default.
• W3205094911 hasConcept C33923547 @default.
• W3205094911 hasConcept C41008148 @default.
• W3205094911 hasConcept C41895202 @default.
• W3205094911 hasConcept C50644808 @default.
• W3205094911 hasConcept C51955184 @default.
• W3205094911 hasConcept C70710897 @default.
• W3205094911 hasConcept C78045399 @default.
• W3205094911 hasConceptScore W3205094911C108583219 @default.
• W3205094911 hasConceptScore W3205094911C119857082 @default.
• W3205094911 hasConceptScore W3205094911C121332964 @default.
• W3205094911 hasConceptScore W3205094911C121864883 @default.
• W3205094911 hasConceptScore W3205094911C126255220 @default.
• W3205094911 hasConceptScore W3205094911C134306372 @default.
• W3205094911 hasConceptScore W3205094911C138885662 @default.
• W3205094911 hasConceptScore W3205094911C154945302 @default.
• W3205094911 hasConceptScore W3205094911C155032097 @default.
• W3205094911 hasConceptScore W3205094911C206688291 @default.
• W3205094911 hasConceptScore W3205094911C2776401178 @default.
• W3205094911 hasConceptScore W3205094911C28826006 @default.
• W3205094911 hasConceptScore W3205094911C33923547 @default.
• W3205094911 hasConceptScore W3205094911C41008148 @default.
• W3205094911 hasConceptScore W3205094911C41895202 @default.
• W3205094911 hasConceptScore W3205094911C50644808 @default.
• W3205094911 hasConceptScore W3205094911C51955184 @default.

• next