FRINK Linked Data Fragments server

Matches in SemOpenAlex for { <https://semopenalex.org/work/W3102093079> ?p ?o ?g. }

• W3102093079 abstract "Neural network training is computationally and memory intensive. Sparse training can reduce the burden on emerging hardware platforms designed to accelerate sparse computations, but it can affect network convergence. In this work, we propose a novel CNN training algorithm Sparse Weight Activation Training (SWAT). SWAT is more computation and memory-efficient than conventional training. SWAT modifies back-propagation based on the empirical insight that convergence during training tends to be robust to the elimination of (i) small magnitude weights during the forward pass and (ii) both small magnitude weights and activations during the backward pass. We evaluate SWAT on recent CNN architectures such as ResNet, VGG, DenseNet and WideResNet using CIFAR-10, CIFAR-100 and ImageNet datasets. For ResNet-50 on ImageNet SWAT reduces total floating-point operations (FLOPS) during training by 80% resulting in a 3.3$times$ training speedup when run on a simulated sparse learning accelerator representative of emerging platforms while incurring only 1.63% reduction in validation accuracy. Moreover, SWAT reduces memory footprint during the backward pass by 23% to 50% for activations and 50% to 90% for weights." @default.
• W3102093079 created "2020-11-23" @default.
• W3102093079 creator A5026788167 @default.
• W3102093079 creator A5087622291 @default.
• W3102093079 date "2020-01-07" @default.
• W3102093079 modified "2023-09-27" @default.
• W3102093079 title "Sparse Weight Activation Training" @default.
• W3102093079 cites W1522301498 @default.
• W3102093079 cites W1686810756 @default.
• W3102093079 cites W1792966054 @default.
• W3102093079 cites W1902934009 @default.
• W3102093079 cites W1966648469 @default.
• W3102093079 cites W1996641400 @default.
• W3102093079 cites W2048266589 @default.
• W3102093079 cites W2058942559 @default.
• W3102093079 cites W2097117768 @default.
• W3102093079 cites W2108598243 @default.
• W3102093079 cites W2114766824 @default.
• W3102093079 cites W2125389748 @default.
• W3102093079 cites W2130306094 @default.
• W3102093079 cites W2146502635 @default.
• W3102093079 cites W2150871235 @default.
• W3102093079 cites W2183341477 @default.
• W3102093079 cites W2194775991 @default.
• W3102093079 cites W2233116163 @default.
• W3102093079 cites W2304648132 @default.
• W3102093079 cites W2319920447 @default.
• W3102093079 cites W2401231614 @default.
• W3102093079 cites W2405920868 @default.
• W3102093079 cites W2469490737 @default.
• W3102093079 cites W2515385951 @default.
• W3102093079 cites W2516141709 @default.
• W3102093079 cites W2559813832 @default.
• W3102093079 cites W2560017826 @default.
• W3102093079 cites W2565851976 @default.
• W3102093079 cites W2582745083 @default.
• W3102093079 cites W2593245696 @default.
• W3102093079 cites W2606722458 @default.
• W3102093079 cites W2613155248 @default.
• W3102093079 cites W2613718673 @default.
• W3102093079 cites W2619096655 @default.
• W3102093079 cites W2625457103 @default.
• W3102093079 cites W2707890836 @default.
• W3102093079 cites W2752387826 @default.
• W3102093079 cites W2753301142 @default.
• W3102093079 cites W2754581707 @default.
• W3102093079 cites W2766224267 @default.
• W3102093079 cites W2786098160 @default.
• W3102093079 cites W2786548044 @default.
• W3102093079 cites W2787759954 @default.
• W3102093079 cites W2791110811 @default.
• W3102093079 cites W2798445803 @default.
• W3102093079 cites W2798956872 @default.
• W3102093079 cites W2799238091 @default.
• W3102093079 cites W2805003733 @default.
• W3102093079 cites W2808168148 @default.
• W3102093079 cites W2886847195 @default.
• W3102093079 cites W2889797931 @default.
• W3102093079 cites W2890616119 @default.
• W3102093079 cites W2899771611 @default.
• W3102093079 cites W2907886210 @default.
• W3102093079 cites W2915478146 @default.
• W3102093079 cites W2931118404 @default.
• W3102093079 cites W2945908221 @default.
• W3102093079 cites W2948223045 @default.
• W3102093079 cites W2949117887 @default.
• W3102093079 cites W2949159496 @default.
• W3102093079 cites W2950458216 @default.
• W3102093079 cites W2952164265 @default.
• W3102093079 cites W2952344559 @default.
• W3102093079 cites W2952689122 @default.
• W3102093079 cites W2956434358 @default.
• W3102093079 cites W2962821792 @default.
• W3102093079 cites W2962851801 @default.
• W3102093079 cites W2963000224 @default.
• W3102093079 cites W2963094099 @default.
• W3102093079 cites W2963363373 @default.
• W3102093079 cites W2963446712 @default.
• W3102093079 cites W2963616439 @default.
• W3102093079 cites W2963709416 @default.
• W3102093079 cites W2963803379 @default.
• W3102093079 cites W2963828549 @default.
• W3102093079 cites W2963982496 @default.
• W3102093079 cites W2964087376 @default.
• W3102093079 cites W2964152344 @default.
• W3102093079 cites W2964164125 @default.
• W3102093079 cites W2964233199 @default.
• W3102093079 cites W2964261616 @default.
• W3102093079 cites W2964299589 @default.
• W3102093079 cites W2971115712 @default.
• W3102093079 cites W2990172850 @default.
• W3102093079 cites W2994749257 @default.
• W3102093079 cites W2995629810 @default.
• W3102093079 cites W3005273253 @default.
• W3102093079 cites W3035180000 @default.
• W3102093079 cites W3035695586 @default.
• W3102093079 cites W3040749250 @default.
• W3102093079 cites W3118608800 @default.
• W3102093079 cites W39411095 @default.
• W3102093079 cites W54257720 @default.

• next