Matches in SemOpenAlex for { <https://semopenalex.org/work/W4312553254> ?p ?o ?g. }
Showing items 1 to 87 of
87
with 100 items per page.
- W4312553254 abstract "Continual Learning (CL) research typically focuses on tackling the phenomenon of catastrophic forgetting in neural networks. Catastrophic forgetting is associated with an abrupt loss of knowledge previously learned by a model when the task, or more broadly the data distribution, being trained on changes. In supervised learning problems this forgetting, resulting from a change in the model's representation, is typically measured or observed by evaluating the decrease in old task performance. However, a model's representation can change without losing knowledge about prior tasks. In this work we consider the concept of representation forgetting, observed by using the difference in performance of an optimal linear classifier before and after a new task is introduced. Using this tool we revisit a number of standard continual learning benchmarks and observe that, through this lens, model representations trained without any explicit control for forgetting often experience small representation forgetting and can sometimes be comparable to methods which explicitly control for forgetting, especially in longer task sequences. We also show that representation forgetting can lead to new insights on the effect of model capacity and loss function used in continual learning. Based on our results, we show that a simple yet competitive approach is to learn representations continually with standard supervised contrastive learning while constructing prototypes of class samples when queried on old samples. <sup xmlns:mml=http://www.w3.org/1998/Math/MathML xmlns:xlink=http://www.w3.org/1999/xlink>1</sup> <sup xmlns:mml=http://www.w3.org/1998/Math/MathML xmlns:xlink=http://www.w3.org/1999/xlink>1</sup> The code to reproduce our results is publicly available at: https://github.com/rezazzr/Probing-Representation-Forgetting" @default.
- W4312553254 created "2023-01-05" @default.
- W4312553254 creator A5021198356 @default.
- W4312553254 creator A5025113992 @default.
- W4312553254 creator A5047013969 @default.
- W4312553254 creator A5051723523 @default.
- W4312553254 creator A5076419371 @default.
- W4312553254 date "2022-06-01" @default.
- W4312553254 modified "2023-10-04" @default.
- W4312553254 title "Probing Representation Forgetting in Supervised and Unsupervised Continual Learning" @default.
- W4312553254 cites W1682403713 @default.
- W4312553254 cites W2054137409 @default.
- W4312553254 cites W2117539524 @default.
- W4312553254 cites W2152161678 @default.
- W4312553254 cites W2163922914 @default.
- W4312553254 cites W2194775991 @default.
- W4312553254 cites W2466618734 @default.
- W4312553254 cites W2473930607 @default.
- W4312553254 cites W2533598788 @default.
- W4312553254 cites W2554616628 @default.
- W4312553254 cites W2594884675 @default.
- W4312553254 cites W2962707369 @default.
- W4312553254 cites W2964189064 @default.
- W4312553254 cites W3097816393 @default.
- W4312553254 cites W3174917907 @default.
- W4312553254 doi "https://doi.org/10.1109/cvpr52688.2022.01621" @default.
- W4312553254 hasPublicationYear "2022" @default.
- W4312553254 type Work @default.
- W4312553254 citedByCount "3" @default.
- W4312553254 countsByYear W43125532542023 @default.
- W4312553254 crossrefType "proceedings-article" @default.
- W4312553254 hasAuthorship W4312553254A5021198356 @default.
- W4312553254 hasAuthorship W4312553254A5025113992 @default.
- W4312553254 hasAuthorship W4312553254A5047013969 @default.
- W4312553254 hasAuthorship W4312553254A5051723523 @default.
- W4312553254 hasAuthorship W4312553254A5076419371 @default.
- W4312553254 hasBestOaLocation W43125532542 @default.
- W4312553254 hasConcept C119857082 @default.
- W4312553254 hasConcept C154945302 @default.
- W4312553254 hasConcept C15744967 @default.
- W4312553254 hasConcept C162324750 @default.
- W4312553254 hasConcept C17744445 @default.
- W4312553254 hasConcept C180747234 @default.
- W4312553254 hasConcept C187736073 @default.
- W4312553254 hasConcept C199539241 @default.
- W4312553254 hasConcept C2776359362 @default.
- W4312553254 hasConcept C2780451532 @default.
- W4312553254 hasConcept C41008148 @default.
- W4312553254 hasConcept C59404180 @default.
- W4312553254 hasConcept C7149132 @default.
- W4312553254 hasConcept C94625758 @default.
- W4312553254 hasConcept C95012187 @default.
- W4312553254 hasConcept C95623464 @default.
- W4312553254 hasConceptScore W4312553254C119857082 @default.
- W4312553254 hasConceptScore W4312553254C154945302 @default.
- W4312553254 hasConceptScore W4312553254C15744967 @default.
- W4312553254 hasConceptScore W4312553254C162324750 @default.
- W4312553254 hasConceptScore W4312553254C17744445 @default.
- W4312553254 hasConceptScore W4312553254C180747234 @default.
- W4312553254 hasConceptScore W4312553254C187736073 @default.
- W4312553254 hasConceptScore W4312553254C199539241 @default.
- W4312553254 hasConceptScore W4312553254C2776359362 @default.
- W4312553254 hasConceptScore W4312553254C2780451532 @default.
- W4312553254 hasConceptScore W4312553254C41008148 @default.
- W4312553254 hasConceptScore W4312553254C59404180 @default.
- W4312553254 hasConceptScore W4312553254C7149132 @default.
- W4312553254 hasConceptScore W4312553254C94625758 @default.
- W4312553254 hasConceptScore W4312553254C95012187 @default.
- W4312553254 hasConceptScore W4312553254C95623464 @default.
- W4312553254 hasFunder F4320334593 @default.
- W4312553254 hasLocation W43125532541 @default.
- W4312553254 hasLocation W43125532542 @default.
- W4312553254 hasOpenAccess W4312553254 @default.
- W4312553254 hasPrimaryLocation W43125532541 @default.
- W4312553254 hasRelatedWork W1964624709 @default.
- W4312553254 hasRelatedWork W2471138382 @default.
- W4312553254 hasRelatedWork W2556319748 @default.
- W4312553254 hasRelatedWork W2979626747 @default.
- W4312553254 hasRelatedWork W3087493185 @default.
- W4312553254 hasRelatedWork W3158586592 @default.
- W4312553254 hasRelatedWork W3200179079 @default.
- W4312553254 hasRelatedWork W4206762304 @default.
- W4312553254 hasRelatedWork W4281760909 @default.
- W4312553254 hasRelatedWork W4362598752 @default.
- W4312553254 isParatext "false" @default.
- W4312553254 isRetracted "false" @default.
- W4312553254 workType "article" @default.