Matches in SemOpenAlex for { <https://semopenalex.org/work/W4297411542> ?p ?o ?g. }
Showing items 1 to 77 of
77
with 100 items per page.
- W4297411542 abstract "Activation compressed training provides a solution towards reducing the memory cost of training deep neural networks~(DNNs). However, state-of-the-art work combines a search of quantization bit-width with the training, which makes the procedure complicated and less transparent. To this end, we propose a simple and effective method to compress DNN training. Our method is motivated by an instructive observation: DNN backward propagation mainly utilizes the low-frequency component (LFC) of the activation maps, while the majority of memory is for caching the high-frequency component (HFC) during the training. This indicates the HFC of activation maps is highly redundant and compressible during DNN training, which inspires our proposed Dual Activation Precision (DIVISION). During the training, DIVISION preserves the high-precision copy of LFC and compresses the HFC into a light-weight copy with low numerical precision. This can significantly reduce the memory cost without negatively affecting the precision of backward propagation such that DIVISION maintains competitive model accuracy. Experiment results show DIVISION has better comprehensive performance than state-of-the-art methods, including over 10x compression of activation maps and competitive training throughput, without loss of model accuracy." @default.
- W4297411542 created "2022-09-28" @default.
- W4297411542 creator A5002869862 @default.
- W4297411542 creator A5007489034 @default.
- W4297411542 creator A5068477431 @default.
- W4297411542 creator A5075644135 @default.
- W4297411542 creator A5081160261 @default.
- W4297411542 creator A5084497683 @default.
- W4297411542 date "2022-08-04" @default.
- W4297411542 modified "2023-10-16" @default.
- W4297411542 title "DIVISION: Memory Efficient Training via Dual Activation Precision" @default.
- W4297411542 doi "https://doi.org/10.48550/arxiv.2208.04187" @default.
- W4297411542 hasPublicationYear "2022" @default.
- W4297411542 type Work @default.
- W4297411542 citedByCount "0" @default.
- W4297411542 crossrefType "posted-content" @default.
- W4297411542 hasAuthorship W4297411542A5002869862 @default.
- W4297411542 hasAuthorship W4297411542A5007489034 @default.
- W4297411542 hasAuthorship W4297411542A5068477431 @default.
- W4297411542 hasAuthorship W4297411542A5075644135 @default.
- W4297411542 hasAuthorship W4297411542A5081160261 @default.
- W4297411542 hasAuthorship W4297411542A5084497683 @default.
- W4297411542 hasBestOaLocation W42974115421 @default.
- W4297411542 hasConcept C113775141 @default.
- W4297411542 hasConcept C11413529 @default.
- W4297411542 hasConcept C121332964 @default.
- W4297411542 hasConcept C124952713 @default.
- W4297411542 hasConcept C142362112 @default.
- W4297411542 hasConcept C154945302 @default.
- W4297411542 hasConcept C157764524 @default.
- W4297411542 hasConcept C168167062 @default.
- W4297411542 hasConcept C2780980858 @default.
- W4297411542 hasConcept C28855332 @default.
- W4297411542 hasConcept C2984842247 @default.
- W4297411542 hasConcept C33923547 @default.
- W4297411542 hasConcept C41008148 @default.
- W4297411542 hasConcept C50644808 @default.
- W4297411542 hasConcept C555944384 @default.
- W4297411542 hasConcept C60798267 @default.
- W4297411542 hasConcept C76155785 @default.
- W4297411542 hasConcept C94375191 @default.
- W4297411542 hasConcept C97355855 @default.
- W4297411542 hasConceptScore W4297411542C113775141 @default.
- W4297411542 hasConceptScore W4297411542C11413529 @default.
- W4297411542 hasConceptScore W4297411542C121332964 @default.
- W4297411542 hasConceptScore W4297411542C124952713 @default.
- W4297411542 hasConceptScore W4297411542C142362112 @default.
- W4297411542 hasConceptScore W4297411542C154945302 @default.
- W4297411542 hasConceptScore W4297411542C157764524 @default.
- W4297411542 hasConceptScore W4297411542C168167062 @default.
- W4297411542 hasConceptScore W4297411542C2780980858 @default.
- W4297411542 hasConceptScore W4297411542C28855332 @default.
- W4297411542 hasConceptScore W4297411542C2984842247 @default.
- W4297411542 hasConceptScore W4297411542C33923547 @default.
- W4297411542 hasConceptScore W4297411542C41008148 @default.
- W4297411542 hasConceptScore W4297411542C50644808 @default.
- W4297411542 hasConceptScore W4297411542C555944384 @default.
- W4297411542 hasConceptScore W4297411542C60798267 @default.
- W4297411542 hasConceptScore W4297411542C76155785 @default.
- W4297411542 hasConceptScore W4297411542C94375191 @default.
- W4297411542 hasConceptScore W4297411542C97355855 @default.
- W4297411542 hasLocation W42974115421 @default.
- W4297411542 hasOpenAccess W4297411542 @default.
- W4297411542 hasPrimaryLocation W42974115421 @default.
- W4297411542 hasRelatedWork W2786771851 @default.
- W4297411542 hasRelatedWork W2898755250 @default.
- W4297411542 hasRelatedWork W2909808664 @default.
- W4297411542 hasRelatedWork W2945739951 @default.
- W4297411542 hasRelatedWork W2989726971 @default.
- W4297411542 hasRelatedWork W3006769269 @default.
- W4297411542 hasRelatedWork W4200629964 @default.
- W4297411542 hasRelatedWork W4221038325 @default.
- W4297411542 hasRelatedWork W4293053895 @default.
- W4297411542 hasRelatedWork W4308650821 @default.
- W4297411542 isParatext "false" @default.
- W4297411542 isRetracted "false" @default.
- W4297411542 workType "article" @default.