Matches in SemOpenAlex for { <https://semopenalex.org/work/W3190534097> ?p ?o ?g. }
Showing items 1 to 97 of
97
with 100 items per page.
- W3190534097 endingPage "2022" @default.
- W3190534097 startingPage "2010" @default.
- W3190534097 abstract "It is widely acknowledged that biological intelligence is capable of learning continually without forgetting previously learned skills. Unfortunately, it has been widely observed that many artificial intelligence techniques, especially (deep) neural network (NN)-based ones, suffer from catastrophic forgetting problem, which severely forgets previous tasks when learning a new one. How to train NNs without catastrophic forgetting, which is termed continual learning, is emerging as a frontier topic and attracting considerable research interest. Inspired by memory replay and synaptic consolidation mechanism in brain, in this article, we propose a novel and simple framework termed memory recall (MeRec) for continual learning with deep NNs. In particular, we first analyze the feature stability across tasks in NN and show that NN can yield task stable features in certain layers. Then, based on this observation, we use a memory module to keep the feature statistics (mean and std) for each learned task. Based on the memory and statistics, we show that a simple replay strategy with Gaussian distribution-based feature regeneration can recall and recover the knowledge from previous tasks. Together with the weight regularization, MeRec preserves weights learned from previous tasks. Based on this simple framework, MeRec achieved leading performance with extremely small memory budget (only two feature vectors for each class) for continual learning on CIFAR-10 and CIFAR-100 datasets, with at least 50% accuracy drop reduction after several tasks compared to previous state-of-the-art approaches." @default.
- W3190534097 created "2021-08-16" @default.
- W3190534097 creator A5028229824 @default.
- W3190534097 creator A5035519525 @default.
- W3190534097 creator A5058600160 @default.
- W3190534097 creator A5077291122 @default.
- W3190534097 creator A5079002529 @default.
- W3190534097 creator A5080722708 @default.
- W3190534097 date "2022-05-01" @default.
- W3190534097 modified "2023-10-16" @default.
- W3190534097 title "Memory Recall: A Simple Neural Network Training Framework Against Catastrophic Forgetting" @default.
- W3190534097 cites W1677182931 @default.
- W3190534097 cites W2116522068 @default.
- W3190534097 cites W2116640950 @default.
- W3190534097 cites W2117539524 @default.
- W3190534097 cites W2130660124 @default.
- W3190534097 cites W2194775991 @default.
- W3190534097 cites W2240067561 @default.
- W3190534097 cites W2473930607 @default.
- W3190534097 cites W2560647685 @default.
- W3190534097 cites W2766447205 @default.
- W3190534097 cites W2786446225 @default.
- W3190534097 cites W2919115771 @default.
- W3190534097 cites W2963072899 @default.
- W3190534097 cites W2963588172 @default.
- W3190534097 cites W2963788399 @default.
- W3190534097 cites W2964189064 @default.
- W3190534097 cites W2967821093 @default.
- W3190534097 cites W3013325675 @default.
- W3190534097 cites W3030364939 @default.
- W3190534097 cites W3048941751 @default.
- W3190534097 cites W3103800629 @default.
- W3190534097 doi "https://doi.org/10.1109/tnnls.2021.3099700" @default.
- W3190534097 hasPubMedId "https://pubmed.ncbi.nlm.nih.gov/34339377" @default.
- W3190534097 hasPublicationYear "2022" @default.
- W3190534097 type Work @default.
- W3190534097 sameAs 3190534097 @default.
- W3190534097 citedByCount "5" @default.
- W3190534097 countsByYear W31905340972022 @default.
- W3190534097 countsByYear W31905340972023 @default.
- W3190534097 crossrefType "journal-article" @default.
- W3190534097 hasAuthorship W3190534097A5028229824 @default.
- W3190534097 hasAuthorship W3190534097A5035519525 @default.
- W3190534097 hasAuthorship W3190534097A5058600160 @default.
- W3190534097 hasAuthorship W3190534097A5077291122 @default.
- W3190534097 hasAuthorship W3190534097A5079002529 @default.
- W3190534097 hasAuthorship W3190534097A5080722708 @default.
- W3190534097 hasConcept C100660578 @default.
- W3190534097 hasConcept C108583219 @default.
- W3190534097 hasConcept C119857082 @default.
- W3190534097 hasConcept C127413603 @default.
- W3190534097 hasConcept C138885662 @default.
- W3190534097 hasConcept C154945302 @default.
- W3190534097 hasConcept C201995342 @default.
- W3190534097 hasConcept C2776401178 @default.
- W3190534097 hasConcept C2780451532 @default.
- W3190534097 hasConcept C41008148 @default.
- W3190534097 hasConcept C41895202 @default.
- W3190534097 hasConcept C50644808 @default.
- W3190534097 hasConcept C7149132 @default.
- W3190534097 hasConceptScore W3190534097C100660578 @default.
- W3190534097 hasConceptScore W3190534097C108583219 @default.
- W3190534097 hasConceptScore W3190534097C119857082 @default.
- W3190534097 hasConceptScore W3190534097C127413603 @default.
- W3190534097 hasConceptScore W3190534097C138885662 @default.
- W3190534097 hasConceptScore W3190534097C154945302 @default.
- W3190534097 hasConceptScore W3190534097C201995342 @default.
- W3190534097 hasConceptScore W3190534097C2776401178 @default.
- W3190534097 hasConceptScore W3190534097C2780451532 @default.
- W3190534097 hasConceptScore W3190534097C41008148 @default.
- W3190534097 hasConceptScore W3190534097C41895202 @default.
- W3190534097 hasConceptScore W3190534097C50644808 @default.
- W3190534097 hasConceptScore W3190534097C7149132 @default.
- W3190534097 hasFunder F4320321001 @default.
- W3190534097 hasFunder F4320335777 @default.
- W3190534097 hasIssue "5" @default.
- W3190534097 hasLocation W31905340971 @default.
- W3190534097 hasLocation W31905340972 @default.
- W3190534097 hasOpenAccess W3190534097 @default.
- W3190534097 hasPrimaryLocation W31905340971 @default.
- W3190534097 hasRelatedWork W3014300295 @default.
- W3190534097 hasRelatedWork W3164822677 @default.
- W3190534097 hasRelatedWork W4223943233 @default.
- W3190534097 hasRelatedWork W4225161397 @default.
- W3190534097 hasRelatedWork W4250304930 @default.
- W3190534097 hasRelatedWork W4312200629 @default.
- W3190534097 hasRelatedWork W4360585206 @default.
- W3190534097 hasRelatedWork W4364306694 @default.
- W3190534097 hasRelatedWork W4380075502 @default.
- W3190534097 hasRelatedWork W4380086463 @default.
- W3190534097 hasVolume "33" @default.
- W3190534097 isParatext "false" @default.
- W3190534097 isRetracted "false" @default.
- W3190534097 magId "3190534097" @default.
- W3190534097 workType "article" @default.