Matches in SemOpenAlex for { <https://semopenalex.org/work/W4221164080> ?p ?o ?g. }
Showing items 1 to 75 of
75
with 100 items per page.
- W4221164080 abstract "Recently, brain-inspired spiking neuron networks (SNNs) have attracted widespread research interest because of their event-driven and energy-efficient characteristics. Still, it is difficult to efficiently train deep SNNs due to the non-differentiability of its activation function, which disables the typically used gradient descent approaches for traditional artificial neural networks (ANNs). Although the adoption of surrogate gradient (SG) formally allows for the back-propagation of losses, the discrete spiking mechanism actually differentiates the loss landscape of SNNs from that of ANNs, failing the surrogate gradient methods to achieve comparable accuracy as for ANNs. In this paper, we first analyze why the current direct training approach with surrogate gradient results in SNNs with poor generalizability. Then we introduce the temporal efficient training (TET) approach to compensate for the loss of momentum in the gradient descent with SG so that the training process can converge into flatter minima with better generalizability. Meanwhile, we demonstrate that TET improves the temporal scalability of SNN and induces a temporal inheritable training for acceleration. Our method consistently outperforms the SOTA on all reported mainstream datasets, including CIFAR-10/100 and ImageNet. Remarkably on DVS-CIFAR10, we obtained 83$%$ top-1 accuracy, over 10$%$ improvement compared to existing state of the art. Codes are available at url{https://github.com/Gus-Lab/temporal_efficient_training}." @default.
- W4221164080 created "2022-04-03" @default.
- W4221164080 creator A5070499878 @default.
- W4221164080 creator A5071088926 @default.
- W4221164080 creator A5075548674 @default.
- W4221164080 creator A5077789495 @default.
- W4221164080 date "2022-02-24" @default.
- W4221164080 modified "2023-09-26" @default.
- W4221164080 title "Temporal Efficient Training of Spiking Neural Network via Gradient Re-weighting" @default.
- W4221164080 doi "https://doi.org/10.48550/arxiv.2202.11946" @default.
- W4221164080 hasPublicationYear "2022" @default.
- W4221164080 type Work @default.
- W4221164080 citedByCount "1" @default.
- W4221164080 countsByYear W42211640802022 @default.
- W4221164080 crossrefType "posted-content" @default.
- W4221164080 hasAuthorship W4221164080A5070499878 @default.
- W4221164080 hasAuthorship W4221164080A5071088926 @default.
- W4221164080 hasAuthorship W4221164080A5075548674 @default.
- W4221164080 hasAuthorship W4221164080A5077789495 @default.
- W4221164080 hasBestOaLocation W42211640801 @default.
- W4221164080 hasConcept C105795698 @default.
- W4221164080 hasConcept C11731999 @default.
- W4221164080 hasConcept C119857082 @default.
- W4221164080 hasConcept C126838900 @default.
- W4221164080 hasConcept C134306372 @default.
- W4221164080 hasConcept C153180895 @default.
- W4221164080 hasConcept C153258448 @default.
- W4221164080 hasConcept C154945302 @default.
- W4221164080 hasConcept C155032097 @default.
- W4221164080 hasConcept C183115368 @default.
- W4221164080 hasConcept C186633575 @default.
- W4221164080 hasConcept C206688291 @default.
- W4221164080 hasConcept C27158222 @default.
- W4221164080 hasConcept C33923547 @default.
- W4221164080 hasConcept C41008148 @default.
- W4221164080 hasConcept C48044578 @default.
- W4221164080 hasConcept C50644808 @default.
- W4221164080 hasConcept C71924100 @default.
- W4221164080 hasConcept C77088390 @default.
- W4221164080 hasConceptScore W4221164080C105795698 @default.
- W4221164080 hasConceptScore W4221164080C11731999 @default.
- W4221164080 hasConceptScore W4221164080C119857082 @default.
- W4221164080 hasConceptScore W4221164080C126838900 @default.
- W4221164080 hasConceptScore W4221164080C134306372 @default.
- W4221164080 hasConceptScore W4221164080C153180895 @default.
- W4221164080 hasConceptScore W4221164080C153258448 @default.
- W4221164080 hasConceptScore W4221164080C154945302 @default.
- W4221164080 hasConceptScore W4221164080C155032097 @default.
- W4221164080 hasConceptScore W4221164080C183115368 @default.
- W4221164080 hasConceptScore W4221164080C186633575 @default.
- W4221164080 hasConceptScore W4221164080C206688291 @default.
- W4221164080 hasConceptScore W4221164080C27158222 @default.
- W4221164080 hasConceptScore W4221164080C33923547 @default.
- W4221164080 hasConceptScore W4221164080C41008148 @default.
- W4221164080 hasConceptScore W4221164080C48044578 @default.
- W4221164080 hasConceptScore W4221164080C50644808 @default.
- W4221164080 hasConceptScore W4221164080C71924100 @default.
- W4221164080 hasConceptScore W4221164080C77088390 @default.
- W4221164080 hasLocation W42211640801 @default.
- W4221164080 hasLocation W42211640802 @default.
- W4221164080 hasOpenAccess W4221164080 @default.
- W4221164080 hasPrimaryLocation W42211640801 @default.
- W4221164080 hasRelatedWork W1489449076 @default.
- W4221164080 hasRelatedWork W1973966843 @default.
- W4221164080 hasRelatedWork W2082482750 @default.
- W4221164080 hasRelatedWork W2087672058 @default.
- W4221164080 hasRelatedWork W2785715987 @default.
- W4221164080 hasRelatedWork W2791240763 @default.
- W4221164080 hasRelatedWork W3159389381 @default.
- W4221164080 hasRelatedWork W3185486575 @default.
- W4221164080 hasRelatedWork W4221164080 @default.
- W4221164080 hasRelatedWork W4310007453 @default.
- W4221164080 isParatext "false" @default.
- W4221164080 isRetracted "false" @default.
- W4221164080 workType "article" @default.