Matches in SemOpenAlex for { <https://semopenalex.org/work/W3166164410> ?p ?o ?g. }
Showing items 1 to 87 of
87
with 100 items per page.
- W3166164410 endingPage "1813" @default.
- W3166164410 startingPage "1803" @default.
- W3166164410 abstract "The increasing size of neural network models has been critical for improvements in their accuracy, but device memory is not growing at the same rate. This creates fundamental challenges for training neural networks within limited memory environments. In this work, we propose ActNN, a memory-efficient training framework that stores randomly quantized activations for back propagation. We prove the convergence of ActNN for general network architectures, and we characterize the impact of quantization on the convergence via an exact expression for the gradient variance. Using our theory, we propose novel mixed-precision quantization strategies that exploit the activation's heterogeneity across feature dimensions, samples, and layers. These techniques can be readily applied to existing dynamic graph frameworks, such as PyTorch, simply by substituting the layers. We evaluate ActNN on mainstream computer vision models for classification, detection, and segmentation tasks. On all these tasks, ActNN compresses the activation to 2 bits on average, with negligible accuracy loss. ActNN reduces the memory footprint of the activation by 12x, and it enables training with a 6.6x to 14x larger batch size." @default.
- W3166164410 created "2021-06-22" @default.
- W3166164410 creator A5004096199 @default.
- W3166164410 creator A5020580195 @default.
- W3166164410 creator A5033006662 @default.
- W3166164410 creator A5033384149 @default.
- W3166164410 creator A5041920173 @default.
- W3166164410 creator A5061339425 @default.
- W3166164410 creator A5072427753 @default.
- W3166164410 date "2021-07-18" @default.
- W3166164410 modified "2023-09-22" @default.
- W3166164410 title "ActNN: Reducing Training Memory Footprint via 2-Bit Activation Compressed Training" @default.
- W3166164410 hasPublicationYear "2021" @default.
- W3166164410 type Work @default.
- W3166164410 sameAs 3166164410 @default.
- W3166164410 citedByCount "0" @default.
- W3166164410 crossrefType "proceedings-article" @default.
- W3166164410 hasAuthorship W3166164410A5004096199 @default.
- W3166164410 hasAuthorship W3166164410A5020580195 @default.
- W3166164410 hasAuthorship W3166164410A5033006662 @default.
- W3166164410 hasAuthorship W3166164410A5033384149 @default.
- W3166164410 hasAuthorship W3166164410A5041920173 @default.
- W3166164410 hasAuthorship W3166164410A5061339425 @default.
- W3166164410 hasAuthorship W3166164410A5072427753 @default.
- W3166164410 hasConcept C111919701 @default.
- W3166164410 hasConcept C113775141 @default.
- W3166164410 hasConcept C11413529 @default.
- W3166164410 hasConcept C132943942 @default.
- W3166164410 hasConcept C151730666 @default.
- W3166164410 hasConcept C154945302 @default.
- W3166164410 hasConcept C155032097 @default.
- W3166164410 hasConcept C162324750 @default.
- W3166164410 hasConcept C165696696 @default.
- W3166164410 hasConcept C2777303404 @default.
- W3166164410 hasConcept C28855332 @default.
- W3166164410 hasConcept C38652104 @default.
- W3166164410 hasConcept C41008148 @default.
- W3166164410 hasConcept C50522688 @default.
- W3166164410 hasConcept C50644808 @default.
- W3166164410 hasConcept C74912251 @default.
- W3166164410 hasConcept C86803240 @default.
- W3166164410 hasConceptScore W3166164410C111919701 @default.
- W3166164410 hasConceptScore W3166164410C113775141 @default.
- W3166164410 hasConceptScore W3166164410C11413529 @default.
- W3166164410 hasConceptScore W3166164410C132943942 @default.
- W3166164410 hasConceptScore W3166164410C151730666 @default.
- W3166164410 hasConceptScore W3166164410C154945302 @default.
- W3166164410 hasConceptScore W3166164410C155032097 @default.
- W3166164410 hasConceptScore W3166164410C162324750 @default.
- W3166164410 hasConceptScore W3166164410C165696696 @default.
- W3166164410 hasConceptScore W3166164410C2777303404 @default.
- W3166164410 hasConceptScore W3166164410C28855332 @default.
- W3166164410 hasConceptScore W3166164410C38652104 @default.
- W3166164410 hasConceptScore W3166164410C41008148 @default.
- W3166164410 hasConceptScore W3166164410C50522688 @default.
- W3166164410 hasConceptScore W3166164410C50644808 @default.
- W3166164410 hasConceptScore W3166164410C74912251 @default.
- W3166164410 hasConceptScore W3166164410C86803240 @default.
- W3166164410 hasLocation W31661644101 @default.
- W3166164410 hasOpenAccess W3166164410 @default.
- W3166164410 hasPrimaryLocation W31661644101 @default.
- W3166164410 hasRelatedWork W2748818695 @default.
- W3166164410 hasRelatedWork W2750958979 @default.
- W3166164410 hasRelatedWork W2798332276 @default.
- W3166164410 hasRelatedWork W2897067384 @default.
- W3166164410 hasRelatedWork W2905117322 @default.
- W3166164410 hasRelatedWork W2912890062 @default.
- W3166164410 hasRelatedWork W2916954108 @default.
- W3166164410 hasRelatedWork W2963480671 @default.
- W3166164410 hasRelatedWork W2963521187 @default.
- W3166164410 hasRelatedWork W2965705320 @default.
- W3166164410 hasRelatedWork W3000788108 @default.
- W3166164410 hasRelatedWork W3021848436 @default.
- W3166164410 hasRelatedWork W3035718760 @default.
- W3166164410 hasRelatedWork W3038013679 @default.
- W3166164410 hasRelatedWork W3091445043 @default.
- W3166164410 hasRelatedWork W3119699820 @default.
- W3166164410 hasRelatedWork W3157971608 @default.
- W3166164410 hasRelatedWork W3175566627 @default.
- W3166164410 hasRelatedWork W3183224688 @default.
- W3166164410 hasRelatedWork W3199269302 @default.
- W3166164410 isParatext "false" @default.
- W3166164410 isRetracted "false" @default.
- W3166164410 magId "3166164410" @default.
- W3166164410 workType "article" @default.