Matches in SemOpenAlex for { <https://semopenalex.org/work/W4303833184> ?p ?o ?g. }
Showing items 1 to 77 of
77
with 100 items per page.
- W4303833184 abstract "We present and review an algorithmic and theoretical framework for improving neural network architecture design via momentum. As case studies, we consider how momentum can improve the architecture design for recurrent neural networks (RNNs), neural ordinary differential equations (ODEs), and transformers. We show that integrating momentum into neural network architectures has several remarkable theoretical and empirical benefits, including 1) integrating momentum into RNNs and neural ODEs can overcome the vanishing gradient issues in training RNNs and neural ODEs, resulting in effective learning long-term dependencies. 2) momentum in neural ODEs can reduce the stiffness of the ODE dynamics, which significantly enhances the computational efficiency in training and testing. 3) momentum can improve the efficiency and accuracy of transformers." @default.
- W4303833184 created "2022-10-09" @default.
- W4303833184 creator A5002037883 @default.
- W4303833184 creator A5036026030 @default.
- W4303833184 creator A5063346108 @default.
- W4303833184 creator A5091186466 @default.
- W4303833184 date "2021-10-13" @default.
- W4303833184 modified "2023-10-16" @default.
- W4303833184 title "How Does Momentum Benefit Deep Neural Networks Architecture Design? A Few Case Studies" @default.
- W4303833184 doi "https://doi.org/10.48550/arxiv.2110.07034" @default.
- W4303833184 hasPublicationYear "2021" @default.
- W4303833184 type Work @default.
- W4303833184 citedByCount "0" @default.
- W4303833184 crossrefType "posted-content" @default.
- W4303833184 hasAuthorship W4303833184A5002037883 @default.
- W4303833184 hasAuthorship W4303833184A5036026030 @default.
- W4303833184 hasAuthorship W4303833184A5063346108 @default.
- W4303833184 hasAuthorship W4303833184A5091186466 @default.
- W4303833184 hasBestOaLocation W43038331841 @default.
- W4303833184 hasConcept C10138342 @default.
- W4303833184 hasConcept C108583219 @default.
- W4303833184 hasConcept C119599485 @default.
- W4303833184 hasConcept C123657996 @default.
- W4303833184 hasConcept C127413603 @default.
- W4303833184 hasConcept C134306372 @default.
- W4303833184 hasConcept C142362112 @default.
- W4303833184 hasConcept C147168706 @default.
- W4303833184 hasConcept C153349607 @default.
- W4303833184 hasConcept C154945302 @default.
- W4303833184 hasConcept C162324750 @default.
- W4303833184 hasConcept C165801399 @default.
- W4303833184 hasConcept C28826006 @default.
- W4303833184 hasConcept C33923547 @default.
- W4303833184 hasConcept C34862557 @default.
- W4303833184 hasConcept C41008148 @default.
- W4303833184 hasConcept C50644808 @default.
- W4303833184 hasConcept C51544822 @default.
- W4303833184 hasConcept C60718061 @default.
- W4303833184 hasConcept C66322947 @default.
- W4303833184 hasConcept C78045399 @default.
- W4303833184 hasConceptScore W4303833184C10138342 @default.
- W4303833184 hasConceptScore W4303833184C108583219 @default.
- W4303833184 hasConceptScore W4303833184C119599485 @default.
- W4303833184 hasConceptScore W4303833184C123657996 @default.
- W4303833184 hasConceptScore W4303833184C127413603 @default.
- W4303833184 hasConceptScore W4303833184C134306372 @default.
- W4303833184 hasConceptScore W4303833184C142362112 @default.
- W4303833184 hasConceptScore W4303833184C147168706 @default.
- W4303833184 hasConceptScore W4303833184C153349607 @default.
- W4303833184 hasConceptScore W4303833184C154945302 @default.
- W4303833184 hasConceptScore W4303833184C162324750 @default.
- W4303833184 hasConceptScore W4303833184C165801399 @default.
- W4303833184 hasConceptScore W4303833184C28826006 @default.
- W4303833184 hasConceptScore W4303833184C33923547 @default.
- W4303833184 hasConceptScore W4303833184C34862557 @default.
- W4303833184 hasConceptScore W4303833184C41008148 @default.
- W4303833184 hasConceptScore W4303833184C50644808 @default.
- W4303833184 hasConceptScore W4303833184C51544822 @default.
- W4303833184 hasConceptScore W4303833184C60718061 @default.
- W4303833184 hasConceptScore W4303833184C66322947 @default.
- W4303833184 hasConceptScore W4303833184C78045399 @default.
- W4303833184 hasLocation W43038331841 @default.
- W4303833184 hasOpenAccess W4303833184 @default.
- W4303833184 hasPrimaryLocation W43038331841 @default.
- W4303833184 hasRelatedWork W2810270255 @default.
- W4303833184 hasRelatedWork W2958796454 @default.
- W4303833184 hasRelatedWork W2971278153 @default.
- W4303833184 hasRelatedWork W3005641657 @default.
- W4303833184 hasRelatedWork W3033710643 @default.
- W4303833184 hasRelatedWork W3205535720 @default.
- W4303833184 hasRelatedWork W4226129964 @default.
- W4303833184 hasRelatedWork W4285428721 @default.
- W4303833184 hasRelatedWork W4303833184 @default.
- W4303833184 hasRelatedWork W4320521253 @default.
- W4303833184 isParatext "false" @default.
- W4303833184 isRetracted "false" @default.
- W4303833184 workType "article" @default.