Matches in SemOpenAlex for { <https://semopenalex.org/work/W4280557002> ?p ?o ?g. }
Showing items 1 to 75 of
75
with 100 items per page.
- W4280557002 abstract "Empowered by the backpropagation (BP) algorithm, deep neural networks have dominated the race in solving various cognitive tasks. The restricted training pattern in the standard BP requires end-to-end error propagation, causing large memory cost and prohibiting model parallelization. Existing local training methods aim to resolve the training obstacle by completely cutting off the backward path between modules and isolating their gradients to reduce memory cost and accelerate the training process. These methods prevent errors from flowing between modules and hence information exchange, resulting in inferior performance. This work proposes a novel local training algorithm, BackLink, which introduces inter-module backward dependency and allows errors to flow between modules. The algorithm facilitates information to flow backward along with the network. To preserve the computational advantage of local training, BackLink restricts the error propagation length within the module. Extensive experiments performed in various deep convolutional neural networks demonstrate that our method consistently improves the classification performance of local training algorithms over other methods. For example, in ResNet32 with 16 local modules, our method surpasses the conventional greedy local training method by 4.00% and a recent work by 1.83% in accuracy on CIFAR10, respectively. Analysis of computational costs reveals that small overheads are incurred in GPU memory costs and runtime on multiple GPUs. Our method can lead up to a 79% reduction in memory cost and 52% in simulation runtime in ResNet110 compared to the standard BP. Therefore, our method could create new opportunities for improving training algorithms towards better efficiency and biological plausibility." @default.
- W4280557002 created "2022-05-22" @default.
- W4280557002 creator A5003795739 @default.
- W4280557002 creator A5013660772 @default.
- W4280557002 creator A5078759325 @default.
- W4280557002 creator A5084829415 @default.
- W4280557002 date "2022-05-14" @default.
- W4280557002 modified "2023-09-27" @default.
- W4280557002 title "BackLink: Supervised Local Training with Backward Links" @default.
- W4280557002 doi "https://doi.org/10.48550/arxiv.2205.07141" @default.
- W4280557002 hasPublicationYear "2022" @default.
- W4280557002 type Work @default.
- W4280557002 citedByCount "1" @default.
- W4280557002 countsByYear W42805570022023 @default.
- W4280557002 crossrefType "posted-content" @default.
- W4280557002 hasAuthorship W4280557002A5003795739 @default.
- W4280557002 hasAuthorship W4280557002A5013660772 @default.
- W4280557002 hasAuthorship W4280557002A5078759325 @default.
- W4280557002 hasAuthorship W4280557002A5084829415 @default.
- W4280557002 hasBestOaLocation W42805570021 @default.
- W4280557002 hasConcept C111919701 @default.
- W4280557002 hasConcept C113775141 @default.
- W4280557002 hasConcept C11413529 @default.
- W4280557002 hasConcept C119857082 @default.
- W4280557002 hasConcept C121332964 @default.
- W4280557002 hasConcept C153294291 @default.
- W4280557002 hasConcept C154945302 @default.
- W4280557002 hasConcept C155032097 @default.
- W4280557002 hasConcept C17744445 @default.
- W4280557002 hasConcept C199360897 @default.
- W4280557002 hasConcept C199539241 @default.
- W4280557002 hasConcept C2776650193 @default.
- W4280557002 hasConcept C2777211547 @default.
- W4280557002 hasConcept C2777735758 @default.
- W4280557002 hasConcept C41008148 @default.
- W4280557002 hasConcept C45374587 @default.
- W4280557002 hasConcept C50644808 @default.
- W4280557002 hasConcept C81363708 @default.
- W4280557002 hasConcept C98045186 @default.
- W4280557002 hasConceptScore W4280557002C111919701 @default.
- W4280557002 hasConceptScore W4280557002C113775141 @default.
- W4280557002 hasConceptScore W4280557002C11413529 @default.
- W4280557002 hasConceptScore W4280557002C119857082 @default.
- W4280557002 hasConceptScore W4280557002C121332964 @default.
- W4280557002 hasConceptScore W4280557002C153294291 @default.
- W4280557002 hasConceptScore W4280557002C154945302 @default.
- W4280557002 hasConceptScore W4280557002C155032097 @default.
- W4280557002 hasConceptScore W4280557002C17744445 @default.
- W4280557002 hasConceptScore W4280557002C199360897 @default.
- W4280557002 hasConceptScore W4280557002C199539241 @default.
- W4280557002 hasConceptScore W4280557002C2776650193 @default.
- W4280557002 hasConceptScore W4280557002C2777211547 @default.
- W4280557002 hasConceptScore W4280557002C2777735758 @default.
- W4280557002 hasConceptScore W4280557002C41008148 @default.
- W4280557002 hasConceptScore W4280557002C45374587 @default.
- W4280557002 hasConceptScore W4280557002C50644808 @default.
- W4280557002 hasConceptScore W4280557002C81363708 @default.
- W4280557002 hasConceptScore W4280557002C98045186 @default.
- W4280557002 hasLocation W42805570021 @default.
- W4280557002 hasLocation W42805570022 @default.
- W4280557002 hasOpenAccess W4280557002 @default.
- W4280557002 hasPrimaryLocation W42805570021 @default.
- W4280557002 hasRelatedWork W2337926734 @default.
- W4280557002 hasRelatedWork W2362189222 @default.
- W4280557002 hasRelatedWork W2391384657 @default.
- W4280557002 hasRelatedWork W2435007635 @default.
- W4280557002 hasRelatedWork W2980363492 @default.
- W4280557002 hasRelatedWork W3021430260 @default.
- W4280557002 hasRelatedWork W3027997911 @default.
- W4280557002 hasRelatedWork W4225307033 @default.
- W4280557002 hasRelatedWork W4287776258 @default.
- W4280557002 hasRelatedWork W1629725936 @default.
- W4280557002 isParatext "false" @default.
- W4280557002 isRetracted "false" @default.
- W4280557002 workType "article" @default.