FRINK Linked Data Fragments server

Matches in SemOpenAlex for { <https://semopenalex.org/work/W2891369367> ?p ?o ?g. }

• W2891369367 abstract "Learning long-term dependencies in extended temporal sequences requires credit assignment to events far back in the past. The most common method for training recurrent neural networks, back-propagation through time (BPTT), requires credit information to be propagated backwards through every single step of the forward computation, potentially over thousands or millions of time steps. This becomes computationally expensive or even infeasible when used with long sequences. Importantly, biological brains are unlikely to perform such detailed reverse replay over very long sequences of internal states (consider days, months, or years.) However, humans are often reminded of past memories or mental states which are associated with the current mental state. We consider the hypothesis that such memory associations between past and present could be used for credit assignment through arbitrarily long sequences, propagating the credit assigned to the current state to the associated past state. Based on this principle, we study a novel algorithm which only back-propagates through a few of these temporal skip connections, realized by a learned attention mechanism that associates current states with relevant past states. We demonstrate in experiments that our method matches or outperforms regular BPTT and truncated BPTT in tasks involving particularly long-term dependencies, but without requiring the biologically implausible backward replay through the whole history of states. Additionally, we demonstrate that the proposed method transfers to longer sequences significantly better than LSTMs trained with BPTT and LSTMs trained with full self-attention." @default.
• W2891369367 created "2018-09-27" @default.
• W2891369367 creator A5016769717 @default.
• W2891369367 creator A5020949970 @default.
• W2891369367 creator A5034412289 @default.
• W2891369367 creator A5035400021 @default.
• W2891369367 creator A5047726287 @default.
• W2891369367 creator A5079745325 @default.
• W2891369367 creator A5086198262 @default.
• W2891369367 date "2018-09-11" @default.
• W2891369367 modified "2023-09-27" @default.
• W2891369367 title "Sparse Attentive Backtracking: Temporal CreditAssignment Through Reminding" @default.
• W2891369367 cites W1011330699 @default.
• W2891369367 cites W1522301498 @default.
• W2891369367 cites W1632114991 @default.
• W2891369367 cites W1736701665 @default.
• W2891369367 cites W1895577753 @default.
• W2891369367 cites W1902237438 @default.
• W2891369367 cites W1968266296 @default.
• W2891369367 cites W2001257668 @default.
• W2891369367 cites W2036591015 @default.
• W2891369367 cites W2057653135 @default.
• W2891369367 cites W2064675550 @default.
• W2891369367 cites W2078294512 @default.
• W2891369367 cites W2079161053 @default.
• W2891369367 cites W2115107366 @default.
• W2891369367 cites W2157447713 @default.
• W2891369367 cites W2194775991 @default.
• W2891369367 cites W2282990287 @default.
• W2891369367 cites W2326533993 @default.
• W2891369367 cites W2327501763 @default.
• W2891369367 cites W2507425266 @default.
• W2891369367 cites W2511730936 @default.
• W2891369367 cites W2530887700 @default.
• W2891369367 cites W2575842049 @default.
• W2891369367 cites W2604086949 @default.
• W2891369367 cites W2626778328 @default.
• W2891369367 cites W2963042606 @default.
• W2891369367 cites W2964308564 @default.
• W2891369367 cites W3037787599 @default.
• W2891369367 hasPublicationYear "2018" @default.
• W2891369367 type Work @default.
• W2891369367 sameAs 2891369367 @default.
• W2891369367 citedByCount "2" @default.
• W2891369367 countsByYear W28913693672018 @default.
• W2891369367 countsByYear W28913693672019 @default.
• W2891369367 crossrefType "posted-content" @default.
• W2891369367 hasAuthorship W2891369367A5016769717 @default.
• W2891369367 hasAuthorship W2891369367A5020949970 @default.
• W2891369367 hasAuthorship W2891369367A5034412289 @default.
• W2891369367 hasAuthorship W2891369367A5035400021 @default.
• W2891369367 hasAuthorship W2891369367A5047726287 @default.
• W2891369367 hasAuthorship W2891369367A5079745325 @default.
• W2891369367 hasAuthorship W2891369367A5086198262 @default.
• W2891369367 hasConcept C11413529 @default.
• W2891369367 hasConcept C119857082 @default.
• W2891369367 hasConcept C121332964 @default.
• W2891369367 hasConcept C133488467 @default.
• W2891369367 hasConcept C147168706 @default.
• W2891369367 hasConcept C154945302 @default.
• W2891369367 hasConcept C156884757 @default.
• W2891369367 hasConcept C41008148 @default.
• W2891369367 hasConcept C45374587 @default.
• W2891369367 hasConcept C48103436 @default.
• W2891369367 hasConcept C50644808 @default.
• W2891369367 hasConcept C61797465 @default.
• W2891369367 hasConcept C62520636 @default.
• W2891369367 hasConceptScore W2891369367C11413529 @default.
• W2891369367 hasConceptScore W2891369367C119857082 @default.
• W2891369367 hasConceptScore W2891369367C121332964 @default.
• W2891369367 hasConceptScore W2891369367C133488467 @default.
• W2891369367 hasConceptScore W2891369367C147168706 @default.
• W2891369367 hasConceptScore W2891369367C154945302 @default.
• W2891369367 hasConceptScore W2891369367C156884757 @default.
• W2891369367 hasConceptScore W2891369367C41008148 @default.
• W2891369367 hasConceptScore W2891369367C45374587 @default.
• W2891369367 hasConceptScore W2891369367C48103436 @default.
• W2891369367 hasConceptScore W2891369367C50644808 @default.
• W2891369367 hasConceptScore W2891369367C61797465 @default.
• W2891369367 hasConceptScore W2891369367C62520636 @default.
• W2891369367 hasLocation W28913693671 @default.
• W2891369367 hasOpenAccess W2891369367 @default.
• W2891369367 hasPrimaryLocation W28913693671 @default.
• W2891369367 hasRelatedWork W175242119 @default.
• W2891369367 hasRelatedWork W1965417459 @default.
• W2891369367 hasRelatedWork W2128499899 @default.
• W2891369367 hasRelatedWork W2200297111 @default.
• W2891369367 hasRelatedWork W2247148075 @default.
• W2891369367 hasRelatedWork W2289584534 @default.
• W2891369367 hasRelatedWork W2396102635 @default.
• W2891369367 hasRelatedWork W2753847160 @default.
• W2891369367 hasRelatedWork W2767017507 @default.
• W2891369367 hasRelatedWork W2796054047 @default.
• W2891369367 hasRelatedWork W2919347051 @default.
• W2891369367 hasRelatedWork W2951584298 @default.
• W2891369367 hasRelatedWork W2964059481 @default.
• W2891369367 hasRelatedWork W2970065541 @default.
• W2891369367 hasRelatedWork W3011603274 @default.
• W2891369367 hasRelatedWork W3093473748 @default.
• W2891369367 hasRelatedWork W3200515842 @default.

• next