Matches in SemOpenAlex for { <https://semopenalex.org/work/W2566652804> ?p ?o ?g. }
Showing items 1 to 84 of
84
with 100 items per page.
- W2566652804 abstract "The deep learning community has devised a diverse set of methods to make gradient optimization, using large datasets, of large and highly complex models with deeply cascaded nonlinearities, practical. Taken as a whole, these methods constitute a breakthrough, allowing computational structures which are quite wide, very deep, and with an enormous number and variety of free parameters to be effectively optimized. The result now dominates much of practical machine learning, with applications in machine translation, computer vision, and speech recognition. Many of these methods, viewed through the lens of algorithmic differentiation (AD), can be seen as either addressing issues with the gradient itself, or finding ways of achieving increased efficiency using tricks that are AD-related, but not provided by current AD systems.The goal of this paper is to explain not just those methods of most relevance to AD, but also the technical constraints and mindset which led to their discovery. After explaining this context, we present a laundry list of methods developed by the deep learning community. Two of these are discussed in further mathematical detail: a way to dramatically reduce the size of the tape when performing reverse-mode AD on a (theoretically) time-reversible process like an ODE integrator; and a new mathematical insight that allows for the implementation of a stochastic Newton's method." @default.
- W2566652804 created "2017-01-06" @default.
- W2566652804 creator A5009984528 @default.
- W2566652804 creator A5063994783 @default.
- W2566652804 creator A5079140683 @default.
- W2566652804 date "2016-11-10" @default.
- W2566652804 modified "2023-09-27" @default.
- W2566652804 title "Tricks from Deep Learning." @default.
- W2566652804 cites W114517082 @default.
- W2566652804 cites W1498436455 @default.
- W2566652804 cites W1533861849 @default.
- W2566652804 cites W1815076433 @default.
- W2566652804 cites W1841592590 @default.
- W2566652804 cites W1868018859 @default.
- W2566652804 cites W1924770834 @default.
- W2566652804 cites W1994616650 @default.
- W2566652804 cites W2064675550 @default.
- W2566652804 cites W2076063813 @default.
- W2566652804 cites W2101926813 @default.
- W2566652804 cites W2154890045 @default.
- W2566652804 cites W2176412452 @default.
- W2566652804 cites W2769602321 @default.
- W2566652804 cites W2919115771 @default.
- W2566652804 cites W2951650375 @default.
- W2566652804 hasPublicationYear "2016" @default.
- W2566652804 type Work @default.
- W2566652804 sameAs 2566652804 @default.
- W2566652804 citedByCount "4" @default.
- W2566652804 countsByYear W25666528042015 @default.
- W2566652804 countsByYear W25666528042017 @default.
- W2566652804 countsByYear W25666528042021 @default.
- W2566652804 crossrefType "journal-article" @default.
- W2566652804 hasAuthorship W2566652804A5009984528 @default.
- W2566652804 hasAuthorship W2566652804A5063994783 @default.
- W2566652804 hasAuthorship W2566652804A5079140683 @default.
- W2566652804 hasConcept C108583219 @default.
- W2566652804 hasConcept C111919701 @default.
- W2566652804 hasConcept C119857082 @default.
- W2566652804 hasConcept C136197465 @default.
- W2566652804 hasConcept C151730666 @default.
- W2566652804 hasConcept C154945302 @default.
- W2566652804 hasConcept C2778491294 @default.
- W2566652804 hasConcept C2779343474 @default.
- W2566652804 hasConcept C41008148 @default.
- W2566652804 hasConcept C86803240 @default.
- W2566652804 hasConcept C98045186 @default.
- W2566652804 hasConceptScore W2566652804C108583219 @default.
- W2566652804 hasConceptScore W2566652804C111919701 @default.
- W2566652804 hasConceptScore W2566652804C119857082 @default.
- W2566652804 hasConceptScore W2566652804C136197465 @default.
- W2566652804 hasConceptScore W2566652804C151730666 @default.
- W2566652804 hasConceptScore W2566652804C154945302 @default.
- W2566652804 hasConceptScore W2566652804C2778491294 @default.
- W2566652804 hasConceptScore W2566652804C2779343474 @default.
- W2566652804 hasConceptScore W2566652804C41008148 @default.
- W2566652804 hasConceptScore W2566652804C86803240 @default.
- W2566652804 hasConceptScore W2566652804C98045186 @default.
- W2566652804 hasLocation W25666528041 @default.
- W2566652804 hasOpenAccess W2566652804 @default.
- W2566652804 hasPrimaryLocation W25666528041 @default.
- W2566652804 hasRelatedWork W1576998926 @default.
- W2566652804 hasRelatedWork W162273904 @default.
- W2566652804 hasRelatedWork W2105283111 @default.
- W2566652804 hasRelatedWork W2436219157 @default.
- W2566652804 hasRelatedWork W2559355445 @default.
- W2566652804 hasRelatedWork W2738093346 @default.
- W2566652804 hasRelatedWork W2741056065 @default.
- W2566652804 hasRelatedWork W2801866268 @default.
- W2566652804 hasRelatedWork W2951650375 @default.
- W2566652804 hasRelatedWork W2990296674 @default.
- W2566652804 hasRelatedWork W3008954574 @default.
- W2566652804 hasRelatedWork W3032900566 @default.
- W2566652804 hasRelatedWork W3110442989 @default.
- W2566652804 hasRelatedWork W3118928774 @default.
- W2566652804 hasRelatedWork W3138862539 @default.
- W2566652804 hasRelatedWork W3146969938 @default.
- W2566652804 hasRelatedWork W3174668881 @default.
- W2566652804 hasRelatedWork W3196035320 @default.
- W2566652804 hasRelatedWork W33980390 @default.
- W2566652804 hasRelatedWork W88657458 @default.
- W2566652804 isParatext "false" @default.
- W2566652804 isRetracted "false" @default.
- W2566652804 magId "2566652804" @default.
- W2566652804 workType "article" @default.