Matches in SemOpenAlex for { <https://semopenalex.org/work/W2897449544> ?p ?o ?g. }
Showing items 1 to 78 of
78
with 100 items per page.
- W2897449544 abstract "Temporal models based on recurrent neural networks have proven to be quite powerful in a wide variety of applications, including language modeling and speech processing. However, to train these models, one relies on back-propagation through time, which entails unfolding the network over many time steps, making the process of conducting credit assignment considerably more challenging. Furthermore, the nature of back-propagation itself does not permit the use of non-differentiable activation functions and is inherently sequential, making parallelization of the underlying training process very difficult. In this work, we propose the Parallel Temporal Neural Coding Network, a biologically inspired model trained by the local learning algorithm known as Local Representation Alignment, that aims to resolve the difficulties and problems that plague recurrent networks trained by back-propagation through time. Most notably, this architecture requires neither unrolling nor the derivatives of its internal activation functions. We compare our model and learning procedure to other online back-propagation-through-time alternatives (which also tend to be computationally expensive), including real-time recurrent learning, echo state networks, and unbiased online recurrent optimization, and show that it outperforms them on sequence modeling benchmarks such as Bouncing MNIST, a new benchmark we call Bouncing NotMNIST, and Penn Treebank. Notably, our approach can, in some instances, even outperform full back-propagation through time itself as well as variants such as sparse attentive back-tracking. Furthermore, we present promising experimental results that demonstrate our model's ability to conduct zero-shot adaptation." @default.
- W2897449544 created "2018-10-26" @default.
- W2897449544 creator A5001294898 @default.
- W2897449544 creator A5005431144 @default.
- W2897449544 creator A5068199296 @default.
- W2897449544 creator A5084332360 @default.
- W2897449544 date "2018-10-17" @default.
- W2897449544 modified "2023-10-14" @default.
- W2897449544 title "Online Learning of Recurrent Neural Architectures by Locally Aligning Distributed Representations." @default.
- W2897449544 hasPublicationYear "2018" @default.
- W2897449544 type Work @default.
- W2897449544 sameAs 2897449544 @default.
- W2897449544 citedByCount "2" @default.
- W2897449544 countsByYear W28974495442019 @default.
- W2897449544 crossrefType "posted-content" @default.
- W2897449544 hasAuthorship W2897449544A5001294898 @default.
- W2897449544 hasAuthorship W2897449544A5005431144 @default.
- W2897449544 hasAuthorship W2897449544A5068199296 @default.
- W2897449544 hasAuthorship W2897449544A5084332360 @default.
- W2897449544 hasConcept C111919701 @default.
- W2897449544 hasConcept C119857082 @default.
- W2897449544 hasConcept C13280743 @default.
- W2897449544 hasConcept C137293760 @default.
- W2897449544 hasConcept C147168706 @default.
- W2897449544 hasConcept C153083717 @default.
- W2897449544 hasConcept C154945302 @default.
- W2897449544 hasConcept C155032097 @default.
- W2897449544 hasConcept C185798385 @default.
- W2897449544 hasConcept C186644900 @default.
- W2897449544 hasConcept C205649164 @default.
- W2897449544 hasConcept C206134035 @default.
- W2897449544 hasConcept C40506919 @default.
- W2897449544 hasConcept C41008148 @default.
- W2897449544 hasConcept C50644808 @default.
- W2897449544 hasConcept C98045186 @default.
- W2897449544 hasConceptScore W2897449544C111919701 @default.
- W2897449544 hasConceptScore W2897449544C119857082 @default.
- W2897449544 hasConceptScore W2897449544C13280743 @default.
- W2897449544 hasConceptScore W2897449544C137293760 @default.
- W2897449544 hasConceptScore W2897449544C147168706 @default.
- W2897449544 hasConceptScore W2897449544C153083717 @default.
- W2897449544 hasConceptScore W2897449544C154945302 @default.
- W2897449544 hasConceptScore W2897449544C155032097 @default.
- W2897449544 hasConceptScore W2897449544C185798385 @default.
- W2897449544 hasConceptScore W2897449544C186644900 @default.
- W2897449544 hasConceptScore W2897449544C205649164 @default.
- W2897449544 hasConceptScore W2897449544C206134035 @default.
- W2897449544 hasConceptScore W2897449544C40506919 @default.
- W2897449544 hasConceptScore W2897449544C41008148 @default.
- W2897449544 hasConceptScore W2897449544C50644808 @default.
- W2897449544 hasConceptScore W2897449544C98045186 @default.
- W2897449544 hasLocation W28974495441 @default.
- W2897449544 hasOpenAccess W2897449544 @default.
- W2897449544 hasPrimaryLocation W28974495441 @default.
- W2897449544 hasRelatedWork W1486687522 @default.
- W2897449544 hasRelatedWork W169320414 @default.
- W2897449544 hasRelatedWork W2204252256 @default.
- W2897449544 hasRelatedWork W2355715145 @default.
- W2897449544 hasRelatedWork W2479427798 @default.
- W2897449544 hasRelatedWork W2606461407 @default.
- W2897449544 hasRelatedWork W2769713526 @default.
- W2897449544 hasRelatedWork W2789634097 @default.
- W2897449544 hasRelatedWork W2796487725 @default.
- W2897449544 hasRelatedWork W2805673628 @default.
- W2897449544 hasRelatedWork W2904707853 @default.
- W2897449544 hasRelatedWork W2949545198 @default.
- W2897449544 hasRelatedWork W2952569483 @default.
- W2897449544 hasRelatedWork W3006378306 @default.
- W2897449544 hasRelatedWork W3008124487 @default.
- W2897449544 hasRelatedWork W3008314020 @default.
- W2897449544 hasRelatedWork W3087972519 @default.
- W2897449544 hasRelatedWork W3126611227 @default.
- W2897449544 hasRelatedWork W3138564770 @default.
- W2897449544 hasRelatedWork W3173493671 @default.
- W2897449544 isParatext "false" @default.
- W2897449544 isRetracted "false" @default.
- W2897449544 magId "2897449544" @default.
- W2897449544 workType "article" @default.