Matches in SemOpenAlex for { <https://semopenalex.org/work/W4330339722> ?p ?o ?g. }
Showing items 1 to 57 of
57
with 100 items per page.
- W4330339722 abstract "In supervised continual learning, a deep neural network (DNN) is updated with an ever-growing data stream. Unlike the offline setting where data is shuffled, we cannot make any distributional assumptions about the data stream. Ideally, only one pass through the dataset is needed for computational efficiency. However, existing methods are inadequate and make many assumptions that cannot be made for real-world applications, while simultaneously failing to improve computational efficiency. In this paper, we propose a novel online continual learning method, SIESTA based on wake/sleep framework for training, which is well aligned to the needs of on-device learning. The major goal of SIESTA is to advance compute efficient continual learning so that DNNs can be updated efficiently using far less time and energy. The principal innovations of SIESTA are: 1) rapid online updates using a rehearsal-free, backpropagation-free, and data-driven network update rule during its wake phase, and 2) expedited memory consolidation using a compute-restricted rehearsal policy during its sleep phase. For memory efficiency, SIESTA adapts latent rehearsal using memory indexing from REMIND. Compared to REMIND and prior arts, SIESTA is far more computationally efficient, enabling continual learning on ImageNet-1K in under 2.4 hours on a single GPU; moreover, in the augmentation-free setting it matches the performance of the offline learner, a milestone critical to driving adoption of continual learning in real-world applications." @default.
- W4330339722 created "2023-03-22" @default.
- W4330339722 creator A5030503283 @default.
- W4330339722 creator A5043035821 @default.
- W4330339722 creator A5046979072 @default.
- W4330339722 creator A5061779011 @default.
- W4330339722 creator A5089315508 @default.
- W4330339722 date "2023-03-19" @default.
- W4330339722 modified "2023-09-27" @default.
- W4330339722 title "SIESTA: Efficient Online Continual Learning with Sleep" @default.
- W4330339722 doi "https://doi.org/10.48550/arxiv.2303.10725" @default.
- W4330339722 hasPublicationYear "2023" @default.
- W4330339722 type Work @default.
- W4330339722 citedByCount "0" @default.
- W4330339722 crossrefType "posted-content" @default.
- W4330339722 hasAuthorship W4330339722A5030503283 @default.
- W4330339722 hasAuthorship W4330339722A5043035821 @default.
- W4330339722 hasAuthorship W4330339722A5046979072 @default.
- W4330339722 hasAuthorship W4330339722A5061779011 @default.
- W4330339722 hasAuthorship W4330339722A5089315508 @default.
- W4330339722 hasBestOaLocation W43303397221 @default.
- W4330339722 hasConcept C119857082 @default.
- W4330339722 hasConcept C154945302 @default.
- W4330339722 hasConcept C178790620 @default.
- W4330339722 hasConcept C183971685 @default.
- W4330339722 hasConcept C185592680 @default.
- W4330339722 hasConcept C185818568 @default.
- W4330339722 hasConcept C32909587 @default.
- W4330339722 hasConcept C41008148 @default.
- W4330339722 hasConcept C50644808 @default.
- W4330339722 hasConcept C75165309 @default.
- W4330339722 hasConceptScore W4330339722C119857082 @default.
- W4330339722 hasConceptScore W4330339722C154945302 @default.
- W4330339722 hasConceptScore W4330339722C178790620 @default.
- W4330339722 hasConceptScore W4330339722C183971685 @default.
- W4330339722 hasConceptScore W4330339722C185592680 @default.
- W4330339722 hasConceptScore W4330339722C185818568 @default.
- W4330339722 hasConceptScore W4330339722C32909587 @default.
- W4330339722 hasConceptScore W4330339722C41008148 @default.
- W4330339722 hasConceptScore W4330339722C50644808 @default.
- W4330339722 hasConceptScore W4330339722C75165309 @default.
- W4330339722 hasLocation W43303397221 @default.
- W4330339722 hasOpenAccess W4330339722 @default.
- W4330339722 hasPrimaryLocation W43303397221 @default.
- W4330339722 hasRelatedWork W1991645216 @default.
- W4330339722 hasRelatedWork W2122688963 @default.
- W4330339722 hasRelatedWork W2348101466 @default.
- W4330339722 hasRelatedWork W2413339343 @default.
- W4330339722 hasRelatedWork W2912850104 @default.
- W4330339722 hasRelatedWork W2961085424 @default.
- W4330339722 hasRelatedWork W4286629047 @default.
- W4330339722 hasRelatedWork W4306321456 @default.
- W4330339722 hasRelatedWork W4306674287 @default.
- W4330339722 hasRelatedWork W4224009465 @default.
- W4330339722 isParatext "false" @default.
- W4330339722 isRetracted "false" @default.
- W4330339722 workType "article" @default.