Matches in SemOpenAlex for { <https://semopenalex.org/work/W4286850779> ?p ?o ?g. }
Showing items 1 to 80 of
80
with 100 items per page.
- W4286850779 abstract "Artificial neural networks have successfully tackled a large variety of problems by training extremely deep networks via back-propagation. A direct application of back-propagation to spiking neural networks contains biologically implausible components, like the weight transport problem or separate inference and learning phases. Various methods address different components individually, but a complete solution remains intangible. Here, we take an alternate approach that avoids back-propagation and its associated issues entirely. Recent work in deep learning proposed independently training each layer of a network via the information bottleneck (IB). Subsequent studies noted that this layer-wise approach circumvents error propagation across layers, leading to a biologically plausible paradigm. Unfortunately, the IB is computed using a batch of samples. The prior work addresses this with a weight update that only uses two samples (the current and previous sample). Our work takes a different approach by decomposing the weight update into a local and global component. The local component is Hebbian and only depends on the current sample. The global component computes a layer-wise modulatory signal that depends on a batch of samples. We show that this modulatory signal can be learned by an auxiliary circuit with working memory (WM) like a reservoir. Thus, we can use batch sizes greater than two, and the batch size determines the required capacity of the WM. To the best of our knowledge, our rule is the first biologically plausible mechanism to directly couple synaptic updates with a WM of the task. We evaluate our rule on synthetic datasets and image classification datasets like MNIST, and we explore the effect of the WM capacity on learning performance. We hope our work is a first-step towards understanding the mechanistic role of memory in learning." @default.
- W4286850779 created "2022-07-25" @default.
- W4286850779 creator A5052869119 @default.
- W4286850779 creator A5070326393 @default.
- W4286850779 date "2021-11-24" @default.
- W4286850779 modified "2023-09-29" @default.
- W4286850779 title "Information Bottleneck-Based Hebbian Learning Rule Naturally Ties Working Memory and Synaptic Updates" @default.
- W4286850779 doi "https://doi.org/10.48550/arxiv.2111.13187" @default.
- W4286850779 hasPublicationYear "2021" @default.
- W4286850779 type Work @default.
- W4286850779 citedByCount "0" @default.
- W4286850779 crossrefType "posted-content" @default.
- W4286850779 hasAuthorship W4286850779A5052869119 @default.
- W4286850779 hasAuthorship W4286850779A5070326393 @default.
- W4286850779 hasBestOaLocation W42868507791 @default.
- W4286850779 hasConcept C108583219 @default.
- W4286850779 hasConcept C111437709 @default.
- W4286850779 hasConcept C117765406 @default.
- W4286850779 hasConcept C119857082 @default.
- W4286850779 hasConcept C121332964 @default.
- W4286850779 hasConcept C125411270 @default.
- W4286850779 hasConcept C149635348 @default.
- W4286850779 hasConcept C154945302 @default.
- W4286850779 hasConcept C155032097 @default.
- W4286850779 hasConcept C168167062 @default.
- W4286850779 hasConcept C17061570 @default.
- W4286850779 hasConcept C178790620 @default.
- W4286850779 hasConcept C185592680 @default.
- W4286850779 hasConcept C199360897 @default.
- W4286850779 hasConcept C2776214188 @default.
- W4286850779 hasConcept C2779127903 @default.
- W4286850779 hasConcept C2779227376 @default.
- W4286850779 hasConcept C2779843651 @default.
- W4286850779 hasConcept C2780513914 @default.
- W4286850779 hasConcept C41008148 @default.
- W4286850779 hasConcept C50644808 @default.
- W4286850779 hasConcept C66949984 @default.
- W4286850779 hasConcept C97108695 @default.
- W4286850779 hasConcept C97355855 @default.
- W4286850779 hasConceptScore W4286850779C108583219 @default.
- W4286850779 hasConceptScore W4286850779C111437709 @default.
- W4286850779 hasConceptScore W4286850779C117765406 @default.
- W4286850779 hasConceptScore W4286850779C119857082 @default.
- W4286850779 hasConceptScore W4286850779C121332964 @default.
- W4286850779 hasConceptScore W4286850779C125411270 @default.
- W4286850779 hasConceptScore W4286850779C149635348 @default.
- W4286850779 hasConceptScore W4286850779C154945302 @default.
- W4286850779 hasConceptScore W4286850779C155032097 @default.
- W4286850779 hasConceptScore W4286850779C168167062 @default.
- W4286850779 hasConceptScore W4286850779C17061570 @default.
- W4286850779 hasConceptScore W4286850779C178790620 @default.
- W4286850779 hasConceptScore W4286850779C185592680 @default.
- W4286850779 hasConceptScore W4286850779C199360897 @default.
- W4286850779 hasConceptScore W4286850779C2776214188 @default.
- W4286850779 hasConceptScore W4286850779C2779127903 @default.
- W4286850779 hasConceptScore W4286850779C2779227376 @default.
- W4286850779 hasConceptScore W4286850779C2779843651 @default.
- W4286850779 hasConceptScore W4286850779C2780513914 @default.
- W4286850779 hasConceptScore W4286850779C41008148 @default.
- W4286850779 hasConceptScore W4286850779C50644808 @default.
- W4286850779 hasConceptScore W4286850779C66949984 @default.
- W4286850779 hasConceptScore W4286850779C97108695 @default.
- W4286850779 hasConceptScore W4286850779C97355855 @default.
- W4286850779 hasLocation W42868507791 @default.
- W4286850779 hasLocation W42868507792 @default.
- W4286850779 hasOpenAccess W4286850779 @default.
- W4286850779 hasPrimaryLocation W42868507791 @default.
- W4286850779 hasRelatedWork W1520894008 @default.
- W4286850779 hasRelatedWork W1530541224 @default.
- W4286850779 hasRelatedWork W1894248549 @default.
- W4286850779 hasRelatedWork W2134664762 @default.
- W4286850779 hasRelatedWork W2170201788 @default.
- W4286850779 hasRelatedWork W2283131038 @default.
- W4286850779 hasRelatedWork W2544817431 @default.
- W4286850779 hasRelatedWork W2604086949 @default.
- W4286850779 hasRelatedWork W3110559181 @default.
- W4286850779 hasRelatedWork W4280574055 @default.
- W4286850779 isParatext "false" @default.
- W4286850779 isRetracted "false" @default.
- W4286850779 workType "article" @default.