Matches in SemOpenAlex for { <https://semopenalex.org/work/W4383473212> ?p ?o ?g. }
Showing items 1 to 71 of
71
with 100 items per page.
- W4383473212 abstract "Continual learning (CL) aims to incrementally learn different tasks (such as classification) in a non-stationary data stream without forgetting old ones. Most CL works focus on tackling catastrophic forgetting under a learning-from-scratch paradigm. However, with the increasing prominence of foundation models, pre-trained models equipped with informative representations have become available for various downstream requirements. Several CL methods based on pre-trained models have been explored, either utilizing pre-extracted features directly (which makes bridging distribution gaps challenging) or incorporating adaptors (which may be subject to forgetting). In this paper, we propose a concise and effective approach for CL with pre-trained models. Given that forgetting occurs during parameter updating, we contemplate an alternative approach that exploits training-free random projectors and class-prototype accumulation, which thus bypasses the issue. Specifically, we inject a frozen Random Projection layer with nonlinear activation between the pre-trained model's feature representations and output head, which captures interactions between features with expanded dimensionality, providing enhanced linear separability for class-prototype-based CL. We also demonstrate the importance of decorrelating the class-prototypes to reduce the distribution disparity when using pre-trained representations. These techniques prove to be effective and circumvent the problem of forgetting for both class- and domain-incremental continual learning. Compared to previous methods applied to pre-trained ViT-B/16 models, we reduce final error rates by between 10% and 62% on seven class-incremental benchmark datasets, despite not using any rehearsal memory. We conclude that the full potential of pre-trained models for simple, effective, and fast continual learning has not hitherto been fully tapped." @default.
- W4383473212 created "2023-07-07" @default.
- W4383473212 creator A5000677611 @default.
- W4383473212 creator A5028024287 @default.
- W4383473212 creator A5054036938 @default.
- W4383473212 creator A5058061726 @default.
- W4383473212 creator A5092418566 @default.
- W4383473212 date "2023-07-05" @default.
- W4383473212 modified "2023-10-01" @default.
- W4383473212 title "RanPAC: Random Projections and Pre-trained Models for Continual Learning" @default.
- W4383473212 doi "https://doi.org/10.48550/arxiv.2307.02251" @default.
- W4383473212 hasPublicationYear "2023" @default.
- W4383473212 type Work @default.
- W4383473212 citedByCount "0" @default.
- W4383473212 crossrefType "posted-content" @default.
- W4383473212 hasAuthorship W4383473212A5000677611 @default.
- W4383473212 hasAuthorship W4383473212A5028024287 @default.
- W4383473212 hasAuthorship W4383473212A5054036938 @default.
- W4383473212 hasAuthorship W4383473212A5058061726 @default.
- W4383473212 hasAuthorship W4383473212A5092418566 @default.
- W4383473212 hasBestOaLocation W43834732121 @default.
- W4383473212 hasConcept C111030470 @default.
- W4383473212 hasConcept C119857082 @default.
- W4383473212 hasConcept C120665830 @default.
- W4383473212 hasConcept C121332964 @default.
- W4383473212 hasConcept C13280743 @default.
- W4383473212 hasConcept C138885662 @default.
- W4383473212 hasConcept C154945302 @default.
- W4383473212 hasConcept C174348530 @default.
- W4383473212 hasConcept C185798385 @default.
- W4383473212 hasConcept C192209626 @default.
- W4383473212 hasConcept C205649164 @default.
- W4383473212 hasConcept C2776401178 @default.
- W4383473212 hasConcept C2777212361 @default.
- W4383473212 hasConcept C31258907 @default.
- W4383473212 hasConcept C41008148 @default.
- W4383473212 hasConcept C41895202 @default.
- W4383473212 hasConcept C7149132 @default.
- W4383473212 hasConceptScore W4383473212C111030470 @default.
- W4383473212 hasConceptScore W4383473212C119857082 @default.
- W4383473212 hasConceptScore W4383473212C120665830 @default.
- W4383473212 hasConceptScore W4383473212C121332964 @default.
- W4383473212 hasConceptScore W4383473212C13280743 @default.
- W4383473212 hasConceptScore W4383473212C138885662 @default.
- W4383473212 hasConceptScore W4383473212C154945302 @default.
- W4383473212 hasConceptScore W4383473212C174348530 @default.
- W4383473212 hasConceptScore W4383473212C185798385 @default.
- W4383473212 hasConceptScore W4383473212C192209626 @default.
- W4383473212 hasConceptScore W4383473212C205649164 @default.
- W4383473212 hasConceptScore W4383473212C2776401178 @default.
- W4383473212 hasConceptScore W4383473212C2777212361 @default.
- W4383473212 hasConceptScore W4383473212C31258907 @default.
- W4383473212 hasConceptScore W4383473212C41008148 @default.
- W4383473212 hasConceptScore W4383473212C41895202 @default.
- W4383473212 hasConceptScore W4383473212C7149132 @default.
- W4383473212 hasLocation W43834732121 @default.
- W4383473212 hasOpenAccess W4383473212 @default.
- W4383473212 hasPrimaryLocation W43834732121 @default.
- W4383473212 hasRelatedWork W112744582 @default.
- W4383473212 hasRelatedWork W1485630101 @default.
- W4383473212 hasRelatedWork W1807477017 @default.
- W4383473212 hasRelatedWork W2498017833 @default.
- W4383473212 hasRelatedWork W2961085424 @default.
- W4383473212 hasRelatedWork W4224329779 @default.
- W4383473212 hasRelatedWork W4281760909 @default.
- W4383473212 hasRelatedWork W4295936673 @default.
- W4383473212 hasRelatedWork W4306674287 @default.
- W4383473212 hasRelatedWork W4224009465 @default.
- W4383473212 isParatext "false" @default.
- W4383473212 isRetracted "false" @default.
- W4383473212 workType "article" @default.