Matches in SemOpenAlex for { <https://semopenalex.org/work/W2574124562> ?p ?o ?g. }
Showing items 1 to 68 of
68
with 100 items per page.
- W2574124562 abstract "Credit assignment in traditional recurrent neural networks usually involves back-propagating through a long chain of tied weight matrices. The length of this chain scales linearly with the number of time-steps as the same network is run at each time-step. This creates many problems, such as vanishing gradients, that have been well studied. In contrast, a NNEM's architecture recurrent activity doesn't involve a long chain of activity (though some architectures such as the NTM do utilize a traditional recurrent architecture as a controller). Rather, the externally stored embedding vectors are used at each time-step, but no messages are passed from previous time-steps. This means that vanishing gradients aren't a problem, as all of the necessary gradient paths are short. However, these paths are extremely numerous (one per embedding vector in memory) and reused for a very long time (until it leaves the memory). Thus, the forward-pass information of each memory must be stored for the entire duration of the memory. This is problematic as this additional storage far surpasses that of the actual memories, to the extent that large memories on infeasible to back-propagate through in high dimensional settings. One way to get around the need to hold onto forward-pass information is to recalculate the forward-pass whenever gradient information is available. However, if the observations are too large to store in the domain of interest, direct reinstatement of a forward pass cannot occur. Instead, we rely on a learned autoencoder to reinstate the observation, and then use the embedding network to recalculate the forward-pass. Since the recalculated embedding vector is unlikely to perfectly match the one stored in memory, we try out 2 approximations to utilize error gradient w.r.t. the vector in memory." @default.
- W2574124562 created "2017-01-26" @default.
- W2574124562 creator A5056424874 @default.
- W2574124562 date "2017-01-14" @default.
- W2574124562 modified "2023-09-27" @default.
- W2574124562 title "Long Timescale Credit Assignment in NeuralNetworks with External Memory" @default.
- W2574124562 cites W2047057213 @default.
- W2574124562 cites W2069143585 @default.
- W2574124562 cites W2436711315 @default.
- W2574124562 cites W2516591743 @default.
- W2574124562 cites W2530887700 @default.
- W2574124562 cites W2950527759 @default.
- W2574124562 cites W2951004968 @default.
- W2574124562 hasPublicationYear "2017" @default.
- W2574124562 type Work @default.
- W2574124562 sameAs 2574124562 @default.
- W2574124562 citedByCount "0" @default.
- W2574124562 crossrefType "posted-content" @default.
- W2574124562 hasAuthorship W2574124562A5056424874 @default.
- W2574124562 hasConcept C100800780 @default.
- W2574124562 hasConcept C101738243 @default.
- W2574124562 hasConcept C11413529 @default.
- W2574124562 hasConcept C147168706 @default.
- W2574124562 hasConcept C154945302 @default.
- W2574124562 hasConcept C41008148 @default.
- W2574124562 hasConcept C41608201 @default.
- W2574124562 hasConcept C50644808 @default.
- W2574124562 hasConcept C80444323 @default.
- W2574124562 hasConcept C9390403 @default.
- W2574124562 hasConcept C98986596 @default.
- W2574124562 hasConceptScore W2574124562C100800780 @default.
- W2574124562 hasConceptScore W2574124562C101738243 @default.
- W2574124562 hasConceptScore W2574124562C11413529 @default.
- W2574124562 hasConceptScore W2574124562C147168706 @default.
- W2574124562 hasConceptScore W2574124562C154945302 @default.
- W2574124562 hasConceptScore W2574124562C41008148 @default.
- W2574124562 hasConceptScore W2574124562C41608201 @default.
- W2574124562 hasConceptScore W2574124562C50644808 @default.
- W2574124562 hasConceptScore W2574124562C80444323 @default.
- W2574124562 hasConceptScore W2574124562C9390403 @default.
- W2574124562 hasConceptScore W2574124562C98986596 @default.
- W2574124562 hasLocation W25741245621 @default.
- W2574124562 hasOpenAccess W2574124562 @default.
- W2574124562 hasPrimaryLocation W25741245621 @default.
- W2574124562 hasRelatedWork W1736523621 @default.
- W2574124562 hasRelatedWork W2002918753 @default.
- W2574124562 hasRelatedWork W2017747420 @default.
- W2574124562 hasRelatedWork W2183902415 @default.
- W2574124562 hasRelatedWork W2296441908 @default.
- W2574124562 hasRelatedWork W2299079197 @default.
- W2574124562 hasRelatedWork W2396417210 @default.
- W2574124562 hasRelatedWork W2484349907 @default.
- W2574124562 hasRelatedWork W2488793855 @default.
- W2574124562 hasRelatedWork W2490175685 @default.
- W2574124562 hasRelatedWork W2580361661 @default.
- W2574124562 hasRelatedWork W2906310736 @default.
- W2574124562 hasRelatedWork W2913006741 @default.
- W2574124562 hasRelatedWork W2988394319 @default.
- W2574124562 hasRelatedWork W3196756515 @default.
- W2574124562 hasRelatedWork W4399642 @default.
- W2574124562 hasRelatedWork W581264992 @default.
- W2574124562 hasRelatedWork W1704573865 @default.
- W2574124562 hasRelatedWork W1931470425 @default.
- W2574124562 hasRelatedWork W2416626103 @default.
- W2574124562 isParatext "false" @default.
- W2574124562 isRetracted "false" @default.
- W2574124562 magId "2574124562" @default.
- W2574124562 workType "article" @default.