Matches in SemOpenAlex for { <https://semopenalex.org/work/W3208382036> ?p ?o ?g. }
Showing items 1 to 90 of
90
with 100 items per page.
- W3208382036 abstract "A fundamental quest in the theory of deep-learning is to understand the properties of the trajectories in the weight space that a learning algorithm takes. One such property that had very recently been isolated is that of ($S_{rm rel}$), which quantifies the propagation of influence of a sampled data point on the prediction at another data point. In this work, we perform a comprehensive study of local elasticity by providing new theoretical insights and more careful empirical evidence of this property in a variety of settings. Firstly, specific to the classification setting, we suggest a new definition of the original idea of $S_{rm rel}$. Via experiments on state-of-the-art neural networks training on SVHN, CIFAR-10 and CIFAR-100 we demonstrate how our new $S_{rm rel}$ detects the property of the weight updates preferring to make changes in predictions within the same class of the sampled data. Next, we demonstrate via examples of neural nets doing regression that the original $S_{rm rel}$ reveals a $2-$phase behaviour: that their training proceeds via an initial elastic phase when $S_{rm rel}$ changes rapidly and an eventual inelastic phase when $S_{rm rel}$ remains large. Lastly, we give multiple examples of learning via gradient flows for which one can get a closed-form expression of the original $S_{rm rel}$ function. By studying the plots of these derived formulas we given a theoretical demonstration of some of the experimentally detected properties of $S_{rm rel}$ in the regression setting." @default.
- W3208382036 created "2021-11-08" @default.
- W3208382036 creator A5047523675 @default.
- W3208382036 creator A5051989561 @default.
- W3208382036 creator A5084835559 @default.
- W3208382036 date "2021-11-02" @default.
- W3208382036 modified "2023-09-27" @default.
- W3208382036 title "Investigating the locality of neural network training dynamics." @default.
- W3208382036 cites W2335728318 @default.
- W3208382036 cites W2603221039 @default.
- W3208382036 cites W2766678531 @default.
- W3208382036 cites W2899790086 @default.
- W3208382036 cites W2962930448 @default.
- W3208382036 cites W2970890756 @default.
- W3208382036 cites W2973294784 @default.
- W3208382036 cites W2993367001 @default.
- W3208382036 cites W2995015865 @default.
- W3208382036 cites W2995658364 @default.
- W3208382036 cites W3020911106 @default.
- W3208382036 cites W3023028000 @default.
- W3208382036 cites W3118608800 @default.
- W3208382036 hasPublicationYear "2021" @default.
- W3208382036 type Work @default.
- W3208382036 sameAs 3208382036 @default.
- W3208382036 citedByCount "0" @default.
- W3208382036 crossrefType "posted-content" @default.
- W3208382036 hasAuthorship W3208382036A5047523675 @default.
- W3208382036 hasAuthorship W3208382036A5051989561 @default.
- W3208382036 hasAuthorship W3208382036A5084835559 @default.
- W3208382036 hasConcept C111472728 @default.
- W3208382036 hasConcept C11413529 @default.
- W3208382036 hasConcept C119857082 @default.
- W3208382036 hasConcept C138885662 @default.
- W3208382036 hasConcept C14036430 @default.
- W3208382036 hasConcept C154945302 @default.
- W3208382036 hasConcept C189950617 @default.
- W3208382036 hasConcept C2524010 @default.
- W3208382036 hasConcept C2777212361 @default.
- W3208382036 hasConcept C2779808786 @default.
- W3208382036 hasConcept C28719098 @default.
- W3208382036 hasConcept C33923547 @default.
- W3208382036 hasConcept C41008148 @default.
- W3208382036 hasConcept C41895202 @default.
- W3208382036 hasConcept C50644808 @default.
- W3208382036 hasConcept C78458016 @default.
- W3208382036 hasConcept C86803240 @default.
- W3208382036 hasConceptScore W3208382036C111472728 @default.
- W3208382036 hasConceptScore W3208382036C11413529 @default.
- W3208382036 hasConceptScore W3208382036C119857082 @default.
- W3208382036 hasConceptScore W3208382036C138885662 @default.
- W3208382036 hasConceptScore W3208382036C14036430 @default.
- W3208382036 hasConceptScore W3208382036C154945302 @default.
- W3208382036 hasConceptScore W3208382036C189950617 @default.
- W3208382036 hasConceptScore W3208382036C2524010 @default.
- W3208382036 hasConceptScore W3208382036C2777212361 @default.
- W3208382036 hasConceptScore W3208382036C2779808786 @default.
- W3208382036 hasConceptScore W3208382036C28719098 @default.
- W3208382036 hasConceptScore W3208382036C33923547 @default.
- W3208382036 hasConceptScore W3208382036C41008148 @default.
- W3208382036 hasConceptScore W3208382036C41895202 @default.
- W3208382036 hasConceptScore W3208382036C50644808 @default.
- W3208382036 hasConceptScore W3208382036C78458016 @default.
- W3208382036 hasConceptScore W3208382036C86803240 @default.
- W3208382036 hasLocation W32083820361 @default.
- W3208382036 hasOpenAccess W3208382036 @default.
- W3208382036 hasPrimaryLocation W32083820361 @default.
- W3208382036 hasRelatedWork W1914062059 @default.
- W3208382036 hasRelatedWork W2044389125 @default.
- W3208382036 hasRelatedWork W2554736449 @default.
- W3208382036 hasRelatedWork W2619069683 @default.
- W3208382036 hasRelatedWork W2786622092 @default.
- W3208382036 hasRelatedWork W2786689480 @default.
- W3208382036 hasRelatedWork W2794354164 @default.
- W3208382036 hasRelatedWork W2971273154 @default.
- W3208382036 hasRelatedWork W2980610698 @default.
- W3208382036 hasRelatedWork W3013992185 @default.
- W3208382036 hasRelatedWork W3035870802 @default.
- W3208382036 hasRelatedWork W3037367037 @default.
- W3208382036 hasRelatedWork W3039771339 @default.
- W3208382036 hasRelatedWork W3046173836 @default.
- W3208382036 hasRelatedWork W3096471977 @default.
- W3208382036 hasRelatedWork W3170996309 @default.
- W3208382036 hasRelatedWork W3171650258 @default.
- W3208382036 hasRelatedWork W3172178011 @default.
- W3208382036 hasRelatedWork W3197102653 @default.
- W3208382036 hasRelatedWork W811226091 @default.
- W3208382036 isParatext "false" @default.
- W3208382036 isRetracted "false" @default.
- W3208382036 magId "3208382036" @default.
- W3208382036 workType "article" @default.