FRINK Linked Data Fragments server

Matches in SemOpenAlex for { <https://semopenalex.org/work/W2971393380> ?p ?o ?g. }

• W2971393380 abstract "Neural networks enjoy widespread use, but many aspects of their training, representation, and operation are poorly understood. In particular, our view into the training process is limited, with a single scalar loss being the most common viewport into this high-dimensional, dynamic process. We propose a new window into training called Loss Change Allocation (LCA), in which credit for changes to the network loss is conservatively partitioned to the parameters. This measurement is accomplished by decomposing the components of an approximate path integral along the training trajectory using a Runge-Kutta integrator. This rich view shows which parameters are responsible for decreasing or increasing the loss during training, or which parameters or the network's learning, respectively. LCA may be summed over training iterations and/or over neurons, channels, or layers for increasingly coarse views. This new measurement device produces several insights into training. (1) We find that barely over 50% of parameters help during any given iteration. (2) Some entire layers hurt overall, moving on average against the training gradient, a phenomenon we hypothesize may be due to phase lag in an oscillatory training process. (3) Finally, increments in learning proceed in a synchronized manner across layers, often peaking on identical iterations." @default.
• W2971393380 created "2019-09-12" @default.
• W2971393380 creator A5000075355 @default.
• W2971393380 creator A5049659753 @default.
• W2971393380 creator A5062174486 @default.
• W2971393380 creator A5075759362 @default.
• W2971393380 date "2019-09-03" @default.
• W2971393380 modified "2023-10-01" @default.
• W2971393380 title "LCA: Loss Change Allocation for Neural Network Training." @default.
• W2971393380 cites W104184427 @default.
• W2971393380 cites W1677182931 @default.
• W2971393380 cites W1825675169 @default.
• W2971393380 cites W1850240193 @default.
• W2971393380 cites W1899249567 @default.
• W2971393380 cites W1904365287 @default.
• W2971393380 cites W1998583908 @default.
• W2971393380 cites W2112796928 @default.
• W2971393380 cites W2169393322 @default.
• W2971393380 cites W2194775991 @default.
• W2971393380 cites W2399994860 @default.
• W2971393380 cites W2523060838 @default.
• W2971393380 cites W2560647685 @default.
• W2971393380 cites W2593634001 @default.
• W2971393380 cites W2599437901 @default.
• W2971393380 cites W2768607686 @default.
• W2971393380 cites W2789115819 @default.
• W2971393380 cites W2805003733 @default.
• W2971393380 cites W2913190747 @default.
• W2971393380 cites W2943283198 @default.
• W2971393380 cites W2949117887 @default.
• W2971393380 cites W2950500611 @default.
• W2971393380 cites W2952502547 @default.
• W2971393380 cites W2962819303 @default.
• W2971393380 cites W2962835968 @default.
• W2971393380 cites W2962933129 @default.
• W2971393380 cites W2963326517 @default.
• W2971393380 cites W2963382180 @default.
• W2971393380 cites W2963427613 @default.
• W2971393380 cites W2963433607 @default.
• W2971393380 cites W2963477238 @default.
• W2971393380 cites W2963940260 @default.
• W2971393380 cites W2964293126 @default.
• W2971393380 hasPublicationYear "2019" @default.
• W2971393380 type Work @default.
• W2971393380 sameAs 2971393380 @default.
• W2971393380 citedByCount "3" @default.
• W2971393380 countsByYear W29713933802020 @default.
• W2971393380 crossrefType "posted-content" @default.
• W2971393380 hasAuthorship W2971393380A5000075355 @default.
• W2971393380 hasAuthorship W2971393380A5049659753 @default.
• W2971393380 hasAuthorship W2971393380A5062174486 @default.
• W2971393380 hasAuthorship W2971393380A5075759362 @default.
• W2971393380 hasConcept C111919701 @default.
• W2971393380 hasConcept C121332964 @default.
• W2971393380 hasConcept C153294291 @default.
• W2971393380 hasConcept C154945302 @default.
• W2971393380 hasConcept C17744445 @default.
• W2971393380 hasConcept C199539241 @default.
• W2971393380 hasConcept C2524010 @default.
• W2971393380 hasConcept C2776257435 @default.
• W2971393380 hasConcept C2776359362 @default.
• W2971393380 hasConcept C2777211547 @default.
• W2971393380 hasConcept C2777735758 @default.
• W2971393380 hasConcept C2778090530 @default.
• W2971393380 hasConcept C31258907 @default.
• W2971393380 hasConcept C33923547 @default.
• W2971393380 hasConcept C41008148 @default.
• W2971393380 hasConcept C50644808 @default.
• W2971393380 hasConcept C57691317 @default.
• W2971393380 hasConcept C76155785 @default.
• W2971393380 hasConcept C79518650 @default.
• W2971393380 hasConcept C94625758 @default.
• W2971393380 hasConcept C98045186 @default.
• W2971393380 hasConceptScore W2971393380C111919701 @default.
• W2971393380 hasConceptScore W2971393380C121332964 @default.
• W2971393380 hasConceptScore W2971393380C153294291 @default.
• W2971393380 hasConceptScore W2971393380C154945302 @default.
• W2971393380 hasConceptScore W2971393380C17744445 @default.
• W2971393380 hasConceptScore W2971393380C199539241 @default.
• W2971393380 hasConceptScore W2971393380C2524010 @default.
• W2971393380 hasConceptScore W2971393380C2776257435 @default.
• W2971393380 hasConceptScore W2971393380C2776359362 @default.
• W2971393380 hasConceptScore W2971393380C2777211547 @default.
• W2971393380 hasConceptScore W2971393380C2777735758 @default.
• W2971393380 hasConceptScore W2971393380C2778090530 @default.
• W2971393380 hasConceptScore W2971393380C31258907 @default.
• W2971393380 hasConceptScore W2971393380C33923547 @default.
• W2971393380 hasConceptScore W2971393380C41008148 @default.
• W2971393380 hasConceptScore W2971393380C50644808 @default.
• W2971393380 hasConceptScore W2971393380C57691317 @default.
• W2971393380 hasConceptScore W2971393380C76155785 @default.
• W2971393380 hasConceptScore W2971393380C79518650 @default.
• W2971393380 hasConceptScore W2971393380C94625758 @default.
• W2971393380 hasConceptScore W2971393380C98045186 @default.
• W2971393380 hasLocation W29713933801 @default.
• W2971393380 hasOpenAccess W2971393380 @default.
• W2971393380 hasPrimaryLocation W29713933801 @default.
• W2971393380 hasRelatedWork W1487967663 @default.
• W2971393380 hasRelatedWork W1606458877 @default.
• W2971393380 hasRelatedWork W1965513449 @default.

• next