FRINK Linked Data Fragments server

Matches in SemOpenAlex for { <https://semopenalex.org/work/W3213234881> ?p ?o ?g. }

• W3213234881 abstract "Training wide and deep neural networks (DNNs) require large amounts of storage resources such as memory because the intermediate activation data must be saved in the memory during forward propagation and then restored for backward propagation. However, state-of-the-art accelerators such as GPUs are only equipped with very limited memory capacities due to hardware design constraints, which significantly limits the maximum batch size and hence performance speedup when training large-scale DNNs. Traditional memory saving techniques either suffer from performance overhead or are constrained by limited interconnect bandwidth or specific interconnect technology. In this paper, we propose a novel memory-efficient CNN training framework (called COMET) that leverages error-bounded lossy compression to significantly reduce the memory requirement for training, to allow training larger models or to accelerate training. Different from the state-of-the-art solutions that adopt image-based lossy compressors (such as JPEG) to compress the activation data, our framework purposely adopts error-bounded lossy compression with a strict error-controlling mechanism. Specifically, we perform a theoretical analysis on the compression error propagation from the altered activation data to the gradients, and empirically investigate the impact of altered gradients over the training process. Based on these analyses, we optimize the error-bounded lossy compression and propose an adaptive error-bound control scheme for activation data compression. We evaluate our design against state-of-the-art solutions with five widely-adopted CNNs and ImageNet dataset. Experiments demonstrate that our proposed framework can significantly reduce the training memory consumption by up to 13.5X over the baseline training and 1.8X over another state-of-the-art compression-based framework, respectively, with little or no accuracy loss." @default.
• W3213234881 created "2021-11-22" @default.
• W3213234881 creator A5025726299 @default.
• W3213234881 creator A5043209884 @default.
• W3213234881 creator A5044713728 @default.
• W3213234881 creator A5048331724 @default.
• W3213234881 creator A5063703614 @default.
• W3213234881 creator A5073748933 @default.
• W3213234881 creator A5078655479 @default.
• W3213234881 creator A5085489377 @default.
• W3213234881 date "2021-11-18" @default.
• W3213234881 modified "2023-10-18" @default.
• W3213234881 title "COMET: A Novel Memory-Efficient Deep Learning Training Framework by Using Error-Bounded Lossy Compression" @default.
• W3213234881 cites W1528999146 @default.
• W3213234881 cites W1686810756 @default.
• W3213234881 cites W1777016212 @default.
• W3213234881 cites W2052440657 @default.
• W3213234881 cites W2097117768 @default.
• W3213234881 cites W2112197601 @default.
• W3213234881 cites W2117130368 @default.
• W3213234881 cites W2118217749 @default.
• W3213234881 cites W2140196014 @default.
• W3213234881 cites W2155893237 @default.
• W3213234881 cites W2157881433 @default.
• W3213234881 cites W2163605009 @default.
• W3213234881 cites W2168694323 @default.
• W3213234881 cites W2194775991 @default.
• W3213234881 cites W2271840356 @default.
• W3213234881 cites W2338908902 @default.
• W3213234881 cites W2339765813 @default.
• W3213234881 cites W2486202470 @default.
• W3213234881 cites W2523246573 @default.
• W3213234881 cites W2612387305 @default.
• W3213234881 cites W2622263826 @default.
• W3213234881 cites W2625759119 @default.
• W3213234881 cites W2727063667 @default.
• W3213234881 cites W2749988060 @default.
• W3213234881 cites W2757910899 @default.
• W3213234881 cites W2763421725 @default.
• W3213234881 cites W2767239597 @default.
• W3213234881 cites W2774644650 @default.
• W3213234881 cites W2777662428 @default.
• W3213234881 cites W2787998955 @default.
• W3213234881 cites W2891993230 @default.
• W3213234881 cites W2895512264 @default.
• W3213234881 cites W2900167092 @default.
• W3213234881 cites W2922073769 @default.
• W3213234881 cites W2952046647 @default.
• W3213234881 cites W2955425717 @default.
• W3213234881 cites W2962821792 @default.
• W3213234881 cites W2963684275 @default.
• W3213234881 cites W2963803379 @default.
• W3213234881 cites W2964174152 @default.
• W3213234881 cites W2964299589 @default.
• W3213234881 cites W2970971581 @default.
• W3213234881 cites W2972087877 @default.
• W3213234881 cites W2988070836 @default.
• W3213234881 cites W2991040477 @default.
• W3213234881 cites W3043443960 @default.
• W3213234881 cites W3043802286 @default.
• W3213234881 cites W3081844508 @default.
• W3213234881 cites W3103894541 @default.
• W3213234881 cites W3131087743 @default.
• W3213234881 cites W3131767313 @default.
• W3213234881 doi "https://doi.org/10.48550/arxiv.2111.09562" @default.
• W3213234881 hasPublicationYear "2021" @default.
• W3213234881 type Work @default.
• W3213234881 sameAs 3213234881 @default.
• W3213234881 citedByCount "0" @default.
• W3213234881 crossrefType "posted-content" @default.
• W3213234881 hasAuthorship W3213234881A5025726299 @default.
• W3213234881 hasAuthorship W3213234881A5043209884 @default.
• W3213234881 hasAuthorship W3213234881A5044713728 @default.
• W3213234881 hasAuthorship W3213234881A5048331724 @default.
• W3213234881 hasAuthorship W3213234881A5063703614 @default.
• W3213234881 hasAuthorship W3213234881A5073748933 @default.
• W3213234881 hasAuthorship W3213234881A5078655479 @default.
• W3213234881 hasAuthorship W3213234881A5085489377 @default.
• W3213234881 hasBestOaLocation W32132348811 @default.
• W3213234881 hasConcept C113775141 @default.
• W3213234881 hasConcept C11413529 @default.
• W3213234881 hasConcept C115961682 @default.
• W3213234881 hasConcept C127413603 @default.
• W3213234881 hasConcept C134306372 @default.
• W3213234881 hasConcept C13481523 @default.
• W3213234881 hasConcept C154945302 @default.
• W3213234881 hasConcept C165021410 @default.
• W3213234881 hasConcept C171146098 @default.
• W3213234881 hasConcept C173608175 @default.
• W3213234881 hasConcept C198751489 @default.
• W3213234881 hasConcept C25797200 @default.
• W3213234881 hasConcept C33923547 @default.
• W3213234881 hasConcept C34388435 @default.
• W3213234881 hasConcept C40969351 @default.
• W3213234881 hasConcept C41008148 @default.
• W3213234881 hasConcept C511840579 @default.
• W3213234881 hasConcept C68339613 @default.
• W3213234881 hasConcept C78548338 @default.
• W3213234881 hasConcept C81363708 @default.
• W3213234881 hasConcept C9417928 @default.

• next