Matches in SemOpenAlex for { <https://semopenalex.org/work/W4311000186> ?p ?o ?g. }
Showing items 1 to 70 of
70
with 100 items per page.
- W4311000186 abstract "The ability to sequentially learn multiple tasks without forgetting is a key skill of biological brains, whereas it represents a major challenge to the field of deep learning. To avoid catastrophic forgetting, various continual learning (CL) approaches have been devised. However, these usually require discrete task boundaries. This requirement seems biologically implausible and often limits the application of CL methods in the real world where tasks are not always well defined. Here, we take inspiration from neuroscience, where sparse, non-overlapping neuronal representations have been suggested to prevent catastrophic forgetting. As in the brain, we argue that these sparse representations should be chosen on the basis of feed forward (stimulus-specific) as well as top-down (context-specific) information. To implement such selective sparsity, we use a bio-plausible form of hierarchical credit assignment known as Deep Feedback Control (DFC) and combine it with a winner-take-all sparsity mechanism. In addition to sparsity, we introduce lateral recurrent connections within each layer to further protect previously learned representations. We evaluate the new sparse-recurrent version of DFC on the split-MNIST computer vision benchmark and show that only the combination of sparsity and intra-layer recurrent connections improves CL performance with respect to standard backpropagation. Our method achieves similar performance to well-known CL methods, such as Elastic Weight Consolidation and Synaptic Intelligence, without requiring information about task boundaries. Overall, we showcase the idea of adopting computational principles from the brain to derive new, task-free learning algorithms for CL." @default.
- W4311000186 created "2022-12-22" @default.
- W4311000186 creator A5043831077 @default.
- W4311000186 creator A5048432296 @default.
- W4311000186 creator A5085762895 @default.
- W4311000186 creator A5090542476 @default.
- W4311000186 date "2022-12-08" @default.
- W4311000186 modified "2023-10-17" @default.
- W4311000186 title "Bio-Inspired, Task-Free Continual Learning through Activity Regularization" @default.
- W4311000186 doi "https://doi.org/10.48550/arxiv.2212.04316" @default.
- W4311000186 hasPublicationYear "2022" @default.
- W4311000186 type Work @default.
- W4311000186 citedByCount "0" @default.
- W4311000186 crossrefType "posted-content" @default.
- W4311000186 hasAuthorship W4311000186A5043831077 @default.
- W4311000186 hasAuthorship W4311000186A5048432296 @default.
- W4311000186 hasAuthorship W4311000186A5085762895 @default.
- W4311000186 hasAuthorship W4311000186A5090542476 @default.
- W4311000186 hasBestOaLocation W43110001861 @default.
- W4311000186 hasConcept C108583219 @default.
- W4311000186 hasConcept C119857082 @default.
- W4311000186 hasConcept C13280743 @default.
- W4311000186 hasConcept C151730666 @default.
- W4311000186 hasConcept C154945302 @default.
- W4311000186 hasConcept C15744967 @default.
- W4311000186 hasConcept C162324750 @default.
- W4311000186 hasConcept C180747234 @default.
- W4311000186 hasConcept C185798385 @default.
- W4311000186 hasConcept C187736073 @default.
- W4311000186 hasConcept C190502265 @default.
- W4311000186 hasConcept C205649164 @default.
- W4311000186 hasConcept C2779343474 @default.
- W4311000186 hasConcept C2780451532 @default.
- W4311000186 hasConcept C41008148 @default.
- W4311000186 hasConcept C7149132 @default.
- W4311000186 hasConcept C86803240 @default.
- W4311000186 hasConceptScore W4311000186C108583219 @default.
- W4311000186 hasConceptScore W4311000186C119857082 @default.
- W4311000186 hasConceptScore W4311000186C13280743 @default.
- W4311000186 hasConceptScore W4311000186C151730666 @default.
- W4311000186 hasConceptScore W4311000186C154945302 @default.
- W4311000186 hasConceptScore W4311000186C15744967 @default.
- W4311000186 hasConceptScore W4311000186C162324750 @default.
- W4311000186 hasConceptScore W4311000186C180747234 @default.
- W4311000186 hasConceptScore W4311000186C185798385 @default.
- W4311000186 hasConceptScore W4311000186C187736073 @default.
- W4311000186 hasConceptScore W4311000186C190502265 @default.
- W4311000186 hasConceptScore W4311000186C205649164 @default.
- W4311000186 hasConceptScore W4311000186C2779343474 @default.
- W4311000186 hasConceptScore W4311000186C2780451532 @default.
- W4311000186 hasConceptScore W4311000186C41008148 @default.
- W4311000186 hasConceptScore W4311000186C7149132 @default.
- W4311000186 hasConceptScore W4311000186C86803240 @default.
- W4311000186 hasLocation W43110001861 @default.
- W4311000186 hasLocation W43110001862 @default.
- W4311000186 hasOpenAccess W4311000186 @default.
- W4311000186 hasPrimaryLocation W43110001861 @default.
- W4311000186 hasRelatedWork W2590796488 @default.
- W4311000186 hasRelatedWork W2734358244 @default.
- W4311000186 hasRelatedWork W2750384547 @default.
- W4311000186 hasRelatedWork W2809732489 @default.
- W4311000186 hasRelatedWork W2886711096 @default.
- W4311000186 hasRelatedWork W2950475743 @default.
- W4311000186 hasRelatedWork W3046591097 @default.
- W4311000186 hasRelatedWork W3088091256 @default.
- W4311000186 hasRelatedWork W4380078352 @default.
- W4311000186 hasRelatedWork W4386603768 @default.
- W4311000186 isParatext "false" @default.
- W4311000186 isRetracted "false" @default.
- W4311000186 workType "article" @default.