Matches in SemOpenAlex for { <https://semopenalex.org/work/W3044520610> ?p ?o ?g. }
Showing items 1 to 81 of
81
with 100 items per page.
- W3044520610 abstract "We demonstrate 10-40% speedups and memory reduction with Wide ResNets, EfficientNets, and Transformer models, with minimal to no loss in accuracy, using SliceOut---a new dropout scheme designed to take advantage of GPU memory layout. By dropping contiguous sets of units at random, our method preserves the regularization properties of dropout while allowing for more efficient low-level implementation, resulting in training speedups through (1) fast memory access and matrix multiplication of smaller tensors, and (2) memory savings by avoiding allocating memory to zero units in weight gradients and activations. Despite its simplicity, our method is highly effective. We demonstrate its efficacy at scale with Wide ResNets & EfficientNets on CIFAR10/100 and ImageNet, as well as Transformers on the LM1B dataset. These speedups and memory savings in training can lead to $CO_2$ emissions reduction of up to 40% for training large models." @default.
- W3044520610 created "2020-07-29" @default.
- W3044520610 creator A5006200326 @default.
- W3044520610 creator A5029186201 @default.
- W3044520610 creator A5067110406 @default.
- W3044520610 creator A5079288315 @default.
- W3044520610 date "2020-07-21" @default.
- W3044520610 modified "2023-09-27" @default.
- W3044520610 title "SliceOut: Training Transformers and CNNs faster while using less memory." @default.
- W3044520610 cites W1904365287 @default.
- W3044520610 cites W2160660594 @default.
- W3044520610 cites W2599192953 @default.
- W3044520610 cites W2626778328 @default.
- W3044520610 cites W2746314669 @default.
- W3044520610 cites W2750384547 @default.
- W3044520610 cites W2774506233 @default.
- W3044520610 cites W2797328513 @default.
- W3044520610 cites W2804935296 @default.
- W3044520610 cites W2890166761 @default.
- W3044520610 cites W2951714314 @default.
- W3044520610 cites W2952020226 @default.
- W3044520610 cites W2955425717 @default.
- W3044520610 cites W2981540061 @default.
- W3044520610 hasPublicationYear "2020" @default.
- W3044520610 type Work @default.
- W3044520610 sameAs 3044520610 @default.
- W3044520610 citedByCount "2" @default.
- W3044520610 countsByYear W30445206102020 @default.
- W3044520610 countsByYear W30445206102021 @default.
- W3044520610 crossrefType "posted-content" @default.
- W3044520610 hasAuthorship W3044520610A5006200326 @default.
- W3044520610 hasAuthorship W3044520610A5029186201 @default.
- W3044520610 hasAuthorship W3044520610A5067110406 @default.
- W3044520610 hasAuthorship W3044520610A5079288315 @default.
- W3044520610 hasConcept C113775141 @default.
- W3044520610 hasConcept C11413529 @default.
- W3044520610 hasConcept C119599485 @default.
- W3044520610 hasConcept C127413603 @default.
- W3044520610 hasConcept C154945302 @default.
- W3044520610 hasConcept C165801399 @default.
- W3044520610 hasConcept C173608175 @default.
- W3044520610 hasConcept C2776135515 @default.
- W3044520610 hasConcept C41008148 @default.
- W3044520610 hasConcept C66322947 @default.
- W3044520610 hasConceptScore W3044520610C113775141 @default.
- W3044520610 hasConceptScore W3044520610C11413529 @default.
- W3044520610 hasConceptScore W3044520610C119599485 @default.
- W3044520610 hasConceptScore W3044520610C127413603 @default.
- W3044520610 hasConceptScore W3044520610C154945302 @default.
- W3044520610 hasConceptScore W3044520610C165801399 @default.
- W3044520610 hasConceptScore W3044520610C173608175 @default.
- W3044520610 hasConceptScore W3044520610C2776135515 @default.
- W3044520610 hasConceptScore W3044520610C41008148 @default.
- W3044520610 hasConceptScore W3044520610C66322947 @default.
- W3044520610 hasLocation W30445206101 @default.
- W3044520610 hasOpenAccess W3044520610 @default.
- W3044520610 hasPrimaryLocation W30445206101 @default.
- W3044520610 hasRelatedWork W2753618922 @default.
- W3044520610 hasRelatedWork W2774841044 @default.
- W3044520610 hasRelatedWork W2902251695 @default.
- W3044520610 hasRelatedWork W2912759934 @default.
- W3044520610 hasRelatedWork W2913185524 @default.
- W3044520610 hasRelatedWork W2918068383 @default.
- W3044520610 hasRelatedWork W2963118768 @default.
- W3044520610 hasRelatedWork W2963873559 @default.
- W3044520610 hasRelatedWork W2969766737 @default.
- W3044520610 hasRelatedWork W2969868335 @default.
- W3044520610 hasRelatedWork W2971381503 @default.
- W3044520610 hasRelatedWork W2983101505 @default.
- W3044520610 hasRelatedWork W2989916540 @default.
- W3044520610 hasRelatedWork W2998669247 @default.
- W3044520610 hasRelatedWork W3034471139 @default.
- W3044520610 hasRelatedWork W3095639648 @default.
- W3044520610 hasRelatedWork W3129831491 @default.
- W3044520610 hasRelatedWork W3131767313 @default.
- W3044520610 hasRelatedWork W3149140528 @default.
- W3044520610 hasRelatedWork W3158057740 @default.
- W3044520610 isParatext "false" @default.
- W3044520610 isRetracted "false" @default.
- W3044520610 magId "3044520610" @default.
- W3044520610 workType "article" @default.