FRINK Linked Data Fragments server

Matches in SemOpenAlex for { <https://semopenalex.org/work/W3044932337> ?p ?o ?g. }

• W3044932337 abstract "The ubiquity of deep neural networks (DNNs) continues to rise, making them a crucial application class for hardware optimizations. However, detailed profiling and characterization of DNN training remains difficult as these applications often run for hours to days on real hardware. Prior works exploit the iterative nature of DNNs to profile a few training iterations. While such a strategy is sound for networks like convolutional neural networks (CNNs), where the nature of the computation is largely input independent, we observe in this work that this approach is sub-optimal for sequence-based neural networks (SQNNs) such as recurrent neural networks (RNNs). The amount and nature of computations in SQNNs can vary for each input, resulting in heterogeneity across iterations. Thus, arbitrarily selecting a few iterations is insufficient to accurately summarize the behavior of the entire training run. To tackle this challenge, we carefully study the factors that impact SQNN training iterations and identify input sequence length as the key determining factor for variations across iterations. We then use this observation to characterize all iterations of an SQNN training run (requiring no profiling or simulation of the application) and select representative iterations, which we term SeqPoints. We analyze two state-of-the-art SQNNs, DeepSpeech2 and Google's Neural Machine Translation (GNMT), and show that SeqPoints can represent their entire training runs accurately, resulting in geomean errors of only 0.11% and 0.53%, respectively, when projecting overall runtime and 0.13% and 1.50% when projecting speedups due to architectural changes. This high accuracy is achieved while reducing the time needed for profiling by 345x and 214x for the two networks compared to full training runs. As a result, SeqPoint can enable analysis of SQNN training runs in mere minutes instead of hours or days." @default.
• W3044932337 created "2020-07-29" @default.
• W3044932337 creator A5015196366 @default.
• W3044932337 creator A5043935249 @default.
• W3044932337 creator A5047054160 @default.
• W3044932337 creator A5060736283 @default.
• W3044932337 date "2020-07-20" @default.
• W3044932337 modified "2023-09-23" @default.
• W3044932337 title "SeqPoint: Identifying Representative Iterations of Sequence-based Neural Networks" @default.
• W3044932337 cites W1494198834 @default.
• W3044932337 cites W1902930330 @default.
• W3044932337 cites W1969241238 @default.
• W3044932337 cites W2076246374 @default.
• W3044932337 cites W2127218421 @default.
• W3044932337 cites W2150196852 @default.
• W3044932337 cites W2153456949 @default.
• W3044932337 cites W2158924248 @default.
• W3044932337 cites W2193413348 @default.
• W3044932337 cites W2234148183 @default.
• W3044932337 cites W2285660444 @default.
• W3044932337 cites W2474388053 @default.
• W3044932337 cites W2512924740 @default.
• W3044932337 cites W2516141709 @default.
• W3044932337 cites W2525778437 @default.
• W3044932337 cites W2540404261 @default.
• W3044932337 cites W2606722458 @default.
• W3044932337 cites W2657126969 @default.
• W3044932337 cites W2735571801 @default.
• W3044932337 cites W2787181861 @default.
• W3044932337 cites W2794670651 @default.
• W3044932337 cites W2798291715 @default.
• W3044932337 cites W2883283076 @default.
• W3044932337 cites W2883929540 @default.
• W3044932337 cites W2886885214 @default.
• W3044932337 cites W2901073342 @default.
• W3044932337 cites W2905927205 @default.
• W3044932337 cites W2913185524 @default.
• W3044932337 cites W2950580142 @default.
• W3044932337 cites W2963341956 @default.
• W3044932337 cites W2963403868 @default.
• W3044932337 cites W2964265128 @default.
• W3044932337 cites W2964330541 @default.
• W3044932337 cites W2989916540 @default.
• W3044932337 hasPublicationYear "2020" @default.
• W3044932337 type Work @default.
• W3044932337 sameAs 3044932337 @default.
• W3044932337 citedByCount "0" @default.
• W3044932337 crossrefType "posted-content" @default.
• W3044932337 hasAuthorship W3044932337A5015196366 @default.
• W3044932337 hasAuthorship W3044932337A5043935249 @default.
• W3044932337 hasAuthorship W3044932337A5047054160 @default.
• W3044932337 hasAuthorship W3044932337A5060736283 @default.
• W3044932337 hasConcept C11413529 @default.
• W3044932337 hasConcept C119857082 @default.
• W3044932337 hasConcept C147168706 @default.
• W3044932337 hasConcept C154945302 @default.
• W3044932337 hasConcept C165696696 @default.
• W3044932337 hasConcept C187191949 @default.
• W3044932337 hasConcept C199360897 @default.
• W3044932337 hasConcept C2778112365 @default.
• W3044932337 hasConcept C2984842247 @default.
• W3044932337 hasConcept C38652104 @default.
• W3044932337 hasConcept C41008148 @default.
• W3044932337 hasConcept C45374587 @default.
• W3044932337 hasConcept C50644808 @default.
• W3044932337 hasConcept C54355233 @default.
• W3044932337 hasConcept C81363708 @default.
• W3044932337 hasConcept C86803240 @default.
• W3044932337 hasConceptScore W3044932337C11413529 @default.
• W3044932337 hasConceptScore W3044932337C119857082 @default.
• W3044932337 hasConceptScore W3044932337C147168706 @default.
• W3044932337 hasConceptScore W3044932337C154945302 @default.
• W3044932337 hasConceptScore W3044932337C165696696 @default.
• W3044932337 hasConceptScore W3044932337C187191949 @default.
• W3044932337 hasConceptScore W3044932337C199360897 @default.
• W3044932337 hasConceptScore W3044932337C2778112365 @default.
• W3044932337 hasConceptScore W3044932337C2984842247 @default.
• W3044932337 hasConceptScore W3044932337C38652104 @default.
• W3044932337 hasConceptScore W3044932337C41008148 @default.
• W3044932337 hasConceptScore W3044932337C45374587 @default.
• W3044932337 hasConceptScore W3044932337C50644808 @default.
• W3044932337 hasConceptScore W3044932337C54355233 @default.
• W3044932337 hasConceptScore W3044932337C81363708 @default.
• W3044932337 hasConceptScore W3044932337C86803240 @default.
• W3044932337 hasOpenAccess W3044932337 @default.
• W3044932337 hasRelatedWork W2160082767 @default.
• W3044932337 hasRelatedWork W2512629640 @default.
• W3044932337 hasRelatedWork W2618541429 @default.
• W3044932337 hasRelatedWork W2812009592 @default.
• W3044932337 hasRelatedWork W2897067384 @default.
• W3044932337 hasRelatedWork W2902196747 @default.
• W3044932337 hasRelatedWork W2905801789 @default.
• W3044932337 hasRelatedWork W2908530716 @default.
• W3044932337 hasRelatedWork W2948208276 @default.
• W3044932337 hasRelatedWork W2962740476 @default.
• W3044932337 hasRelatedWork W2963919236 @default.
• W3044932337 hasRelatedWork W3021848436 @default.
• W3044932337 hasRelatedWork W3091941622 @default.
• W3044932337 hasRelatedWork W3097320606 @default.
• W3044932337 hasRelatedWork W3129780921 @default.

• next