Matches in SemOpenAlex for { <https://semopenalex.org/work/W2970294425> ?p ?o ?g. }
Showing items 1 to 75 of
75
with 100 items per page.
- W2970294425 endingPage "6006" @default.
- W2970294425 startingPage "5996" @default.
- W2970294425 abstract "Deep neural networks have yielded superior performance in many contemporary applications. However, the gradient computation in a deep model with millions of instances leads to a lengthy training process even with modern GPU/TPU hardware acceleration. In this paper, we propose AutoAssist, a simple framework to accelerate training of a deep neural network. Typically, as the training procedure evolves, the amount of improvement by a stochastic gradient update varies dynamically with the choice of instances in the mini-batch. In AutoAssist, we utilize this fact and design an instance shrinking operation that is used to filter out instances with relatively low marginal improvement to the current model; thus the computationally intensive gradient computations are performed on informative instances as much as possible. Specifically, we train a very lightweight Assistant model jointly with the original deep network, which we refer to as Boss. The Assistant model is designed to gauge the importance of a given instance with respect to the current Boss such that the shrinking operation can be applied in the batch generator. With careful design, we train the Boss and Assistant in a nonblocking and asynchronous fashion such that overhead is minimal. To demonstrate the effectiveness of AutoAssist, we conduct experiments on two contemporary applications: image classification using ResNets with varied number of layers, and neural machine translation using LSTMs, ConvS2S and Transformer models. For each application, we verify that AutoAssist leads to significant reduction in training time; in particular, 30% to 40% of the total operation count can be reduced which leads to faster convergence and a corresponding decrease in training time." @default.
- W2970294425 created "2019-09-05" @default.
- W2970294425 creator A5023183059 @default.
- W2970294425 creator A5063459703 @default.
- W2970294425 creator A5083371477 @default.
- W2970294425 date "2019-05-01" @default.
- W2970294425 modified "2023-09-24" @default.
- W2970294425 title "AutoAssist: A Framework to Accelerate Training of Deep Neural Networks" @default.
- W2970294425 hasPublicationYear "2019" @default.
- W2970294425 type Work @default.
- W2970294425 sameAs 2970294425 @default.
- W2970294425 citedByCount "2" @default.
- W2970294425 countsByYear W29702944252021 @default.
- W2970294425 crossrefType "proceedings-article" @default.
- W2970294425 hasAuthorship W2970294425A5023183059 @default.
- W2970294425 hasAuthorship W2970294425A5063459703 @default.
- W2970294425 hasAuthorship W2970294425A5083371477 @default.
- W2970294425 hasConcept C108583219 @default.
- W2970294425 hasConcept C113775141 @default.
- W2970294425 hasConcept C11413529 @default.
- W2970294425 hasConcept C154945302 @default.
- W2970294425 hasConcept C191897082 @default.
- W2970294425 hasConcept C192562407 @default.
- W2970294425 hasConcept C199360897 @default.
- W2970294425 hasConcept C206688291 @default.
- W2970294425 hasConcept C2777020290 @default.
- W2970294425 hasConcept C2779960059 @default.
- W2970294425 hasConcept C41008148 @default.
- W2970294425 hasConcept C45374587 @default.
- W2970294425 hasConcept C50644808 @default.
- W2970294425 hasConcept C81363708 @default.
- W2970294425 hasConceptScore W2970294425C108583219 @default.
- W2970294425 hasConceptScore W2970294425C113775141 @default.
- W2970294425 hasConceptScore W2970294425C11413529 @default.
- W2970294425 hasConceptScore W2970294425C154945302 @default.
- W2970294425 hasConceptScore W2970294425C191897082 @default.
- W2970294425 hasConceptScore W2970294425C192562407 @default.
- W2970294425 hasConceptScore W2970294425C199360897 @default.
- W2970294425 hasConceptScore W2970294425C206688291 @default.
- W2970294425 hasConceptScore W2970294425C2777020290 @default.
- W2970294425 hasConceptScore W2970294425C2779960059 @default.
- W2970294425 hasConceptScore W2970294425C41008148 @default.
- W2970294425 hasConceptScore W2970294425C45374587 @default.
- W2970294425 hasConceptScore W2970294425C50644808 @default.
- W2970294425 hasConceptScore W2970294425C81363708 @default.
- W2970294425 hasLocation W29702944251 @default.
- W2970294425 hasOpenAccess W2970294425 @default.
- W2970294425 hasPrimaryLocation W29702944251 @default.
- W2970294425 hasRelatedWork W2546681868 @default.
- W2970294425 hasRelatedWork W2617019596 @default.
- W2970294425 hasRelatedWork W2799290373 @default.
- W2970294425 hasRelatedWork W2895139741 @default.
- W2970294425 hasRelatedWork W2905116791 @default.
- W2970294425 hasRelatedWork W2943847545 @default.
- W2970294425 hasRelatedWork W2948678916 @default.
- W2970294425 hasRelatedWork W2963374099 @default.
- W2970294425 hasRelatedWork W3008851394 @default.
- W2970294425 hasRelatedWork W3013134448 @default.
- W2970294425 hasRelatedWork W3089416163 @default.
- W2970294425 hasRelatedWork W3115260384 @default.
- W2970294425 hasRelatedWork W3118182806 @default.
- W2970294425 hasRelatedWork W3131685502 @default.
- W2970294425 hasRelatedWork W3131920822 @default.
- W2970294425 hasRelatedWork W3161311774 @default.
- W2970294425 hasRelatedWork W3169302346 @default.
- W2970294425 hasRelatedWork W3185614262 @default.
- W2970294425 hasRelatedWork W3204507145 @default.
- W2970294425 hasRelatedWork W3213495626 @default.
- W2970294425 hasVolume "32" @default.
- W2970294425 isParatext "false" @default.
- W2970294425 isRetracted "false" @default.
- W2970294425 magId "2970294425" @default.
- W2970294425 workType "article" @default.