Matches in SemOpenAlex for { <https://semopenalex.org/work/W2912180158> ?p ?o ?g. }
Showing items 1 to 99 of
99
with 100 items per page.
- W2912180158 abstract "Efforts to reduce the numerical precision of computations in deep learning training have yielded systems that aggressively quantize weights and activations, yet employ wide high-precision accumulators for partial sums in inner-product operations to preserve the quality of convergence. The absence of any framework to analyze the precision requirements of partial sum accumulations results in conservative design choices. This imposes an upper-bound on the reduction of complexity of multiply-accumulate units. We present a statistical approach to analyze the impact of reduced accumulation precision on deep learning training. Observing that a bad choice for accumulation precision results in loss of information that manifests itself as a reduction in variance in an ensemble of partial sums, we derive a set of equations that relate this variance to the length of accumulation and the minimum number of bits needed for accumulation. We apply our analysis to three benchmark networks: CIFAR-10 ResNet 32, ImageNet ResNet 18 and ImageNet AlexNet. In each case, with accumulation precision set in accordance with our proposed equations, the networks successfully converge to the single precision floating-point baseline. We also show that reducing accumulation precision further degrades the quality of the trained network, proving that our equations produce tight bounds. Overall this analysis enables precise tailoring of computation hardware to the application, yielding area- and power-optimal systems." @default.
- W2912180158 created "2019-02-21" @default.
- W2912180158 creator A5001435691 @default.
- W2912180158 creator A5004766079 @default.
- W2912180158 creator A5047303294 @default.
- W2912180158 creator A5056879485 @default.
- W2912180158 creator A5057014407 @default.
- W2912180158 creator A5078440061 @default.
- W2912180158 creator A5082043392 @default.
- W2912180158 date "2019-01-19" @default.
- W2912180158 modified "2023-09-27" @default.
- W2912180158 title "Accumulation Bit-Width Scaling For Ultra-Low Precision Training Of Deep Networks" @default.
- W2912180158 hasPublicationYear "2019" @default.
- W2912180158 type Work @default.
- W2912180158 sameAs 2912180158 @default.
- W2912180158 citedByCount "0" @default.
- W2912180158 crossrefType "posted-content" @default.
- W2912180158 hasAuthorship W2912180158A5001435691 @default.
- W2912180158 hasAuthorship W2912180158A5004766079 @default.
- W2912180158 hasAuthorship W2912180158A5047303294 @default.
- W2912180158 hasAuthorship W2912180158A5056879485 @default.
- W2912180158 hasAuthorship W2912180158A5057014407 @default.
- W2912180158 hasAuthorship W2912180158A5078440061 @default.
- W2912180158 hasAuthorship W2912180158A5082043392 @default.
- W2912180158 hasConcept C105795698 @default.
- W2912180158 hasConcept C108583219 @default.
- W2912180158 hasConcept C111335779 @default.
- W2912180158 hasConcept C113775141 @default.
- W2912180158 hasConcept C11413529 @default.
- W2912180158 hasConcept C121955636 @default.
- W2912180158 hasConcept C13280743 @default.
- W2912180158 hasConcept C144133560 @default.
- W2912180158 hasConcept C154945302 @default.
- W2912180158 hasConcept C162324750 @default.
- W2912180158 hasConcept C177264268 @default.
- W2912180158 hasConcept C185798385 @default.
- W2912180158 hasConcept C19499675 @default.
- W2912180158 hasConcept C196083921 @default.
- W2912180158 hasConcept C199360897 @default.
- W2912180158 hasConcept C205649164 @default.
- W2912180158 hasConcept C2524010 @default.
- W2912180158 hasConcept C2777303404 @default.
- W2912180158 hasConcept C33923547 @default.
- W2912180158 hasConcept C41008148 @default.
- W2912180158 hasConcept C45374587 @default.
- W2912180158 hasConcept C50522688 @default.
- W2912180158 hasConcept C62644790 @default.
- W2912180158 hasConcept C99844830 @default.
- W2912180158 hasConceptScore W2912180158C105795698 @default.
- W2912180158 hasConceptScore W2912180158C108583219 @default.
- W2912180158 hasConceptScore W2912180158C111335779 @default.
- W2912180158 hasConceptScore W2912180158C113775141 @default.
- W2912180158 hasConceptScore W2912180158C11413529 @default.
- W2912180158 hasConceptScore W2912180158C121955636 @default.
- W2912180158 hasConceptScore W2912180158C13280743 @default.
- W2912180158 hasConceptScore W2912180158C144133560 @default.
- W2912180158 hasConceptScore W2912180158C154945302 @default.
- W2912180158 hasConceptScore W2912180158C162324750 @default.
- W2912180158 hasConceptScore W2912180158C177264268 @default.
- W2912180158 hasConceptScore W2912180158C185798385 @default.
- W2912180158 hasConceptScore W2912180158C19499675 @default.
- W2912180158 hasConceptScore W2912180158C196083921 @default.
- W2912180158 hasConceptScore W2912180158C199360897 @default.
- W2912180158 hasConceptScore W2912180158C205649164 @default.
- W2912180158 hasConceptScore W2912180158C2524010 @default.
- W2912180158 hasConceptScore W2912180158C2777303404 @default.
- W2912180158 hasConceptScore W2912180158C33923547 @default.
- W2912180158 hasConceptScore W2912180158C41008148 @default.
- W2912180158 hasConceptScore W2912180158C45374587 @default.
- W2912180158 hasConceptScore W2912180158C50522688 @default.
- W2912180158 hasConceptScore W2912180158C62644790 @default.
- W2912180158 hasConceptScore W2912180158C99844830 @default.
- W2912180158 hasLocation W29121801581 @default.
- W2912180158 hasOpenAccess W2912180158 @default.
- W2912180158 hasPrimaryLocation W29121801581 @default.
- W2912180158 hasRelatedWork W2584314014 @default.
- W2912180158 hasRelatedWork W2726743825 @default.
- W2912180158 hasRelatedWork W2737685714 @default.
- W2912180158 hasRelatedWork W2763421725 @default.
- W2912180158 hasRelatedWork W2799243248 @default.
- W2912180158 hasRelatedWork W2890616119 @default.
- W2912180158 hasRelatedWork W2890770683 @default.
- W2912180158 hasRelatedWork W2940542208 @default.
- W2912180158 hasRelatedWork W2944508492 @default.
- W2912180158 hasRelatedWork W2946955515 @default.
- W2912180158 hasRelatedWork W2963131426 @default.
- W2912180158 hasRelatedWork W2964472257 @default.
- W2912180158 hasRelatedWork W2994782216 @default.
- W2912180158 hasRelatedWork W3008507360 @default.
- W2912180158 hasRelatedWork W3040522390 @default.
- W2912180158 hasRelatedWork W3120250747 @default.
- W2912180158 hasRelatedWork W3123666279 @default.
- W2912180158 hasRelatedWork W3129181244 @default.
- W2912180158 hasRelatedWork W3133264013 @default.
- W2912180158 hasRelatedWork W3173842498 @default.
- W2912180158 isParatext "false" @default.
- W2912180158 isRetracted "false" @default.
- W2912180158 magId "2912180158" @default.
- W2912180158 workType "article" @default.