Matches in SemOpenAlex for { <https://semopenalex.org/work/W2920121562> ?p ?o ?g. }
- W2920121562 abstract "Recent advances in deep neural networks (DNNs) owe their success to training algorithms that use backpropagation and gradient-descent. Backpropagation, while highly effective on von Neumann architectures, becomes inefficient when scaling to large networks. Commonly referred to as the weight transport problem, each neuron's dependence on the weights and errors located deeper in the network require exhaustive data movement which presents a key problem in enhancing the performance and energy-efficiency of machine-learning hardware. In this work, we propose a bio-plausible alternative to backpropagation drawing from advances in feedback alignment algorithms in which the error computation at a single synapse reduces to the product of three scalar values. Using a sparse feedback matrix, we show that a neuron needs only a fraction of the information previously used by the feedback alignment algorithms. Consequently, memory and compute can be partitioned and distributed whichever way produces the most efficient forward pass so long as a single error can be delivered to each neuron. Our results show orders of magnitude improvement in data movement and $2times$ improvement in multiply-and-accumulate operations over backpropagation. Like previous work, we observe that any variant of feedback alignment suffers significant losses in classification accuracy on deep convolutional neural networks. By transferring trained convolutional layers and training the fully connected layers using direct feedback alignment, we demonstrate that direct feedback alignment can obtain results competitive with backpropagation. Furthermore, we observe that using an extremely sparse feedback matrix, rather than a dense one, results in a small accuracy drop while yielding hardware advantages. All the code and results are available under this https URL." @default.
- W2920121562 created "2019-03-11" @default.
- W2920121562 creator A5016864001 @default.
- W2920121562 creator A5017755967 @default.
- W2920121562 creator A5089938000 @default.
- W2920121562 creator A5091408102 @default.
- W2920121562 date "2019-01-30" @default.
- W2920121562 modified "2023-09-22" @default.
- W2920121562 title "Direct Feedback Alignment with Sparse Connections for Local Learning" @default.
- W2920121562 cites W1570411240 @default.
- W2920121562 cites W1596839545 @default.
- W2920121562 cites W1686810756 @default.
- W2920121562 cites W1855112655 @default.
- W2920121562 cites W2047257909 @default.
- W2920121562 cites W2059148040 @default.
- W2920121562 cites W2108598243 @default.
- W2920121562 cites W2118023920 @default.
- W2920121562 cites W2132729131 @default.
- W2920121562 cites W2138913040 @default.
- W2920121562 cites W2154579312 @default.
- W2920121562 cites W2159110831 @default.
- W2920121562 cites W2163605009 @default.
- W2920121562 cites W2165698076 @default.
- W2920121562 cites W2289252105 @default.
- W2920121562 cites W2402144811 @default.
- W2920121562 cites W2432567885 @default.
- W2920121562 cites W2442974303 @default.
- W2920121562 cites W2482290902 @default.
- W2920121562 cites W2508602506 @default.
- W2920121562 cites W2552737632 @default.
- W2920121562 cites W2594492285 @default.
- W2920121562 cites W2783525259 @default.
- W2920121562 cites W2789805928 @default.
- W2920121562 cites W2793168176 @default.
- W2920121562 cites W2794141774 @default.
- W2920121562 cites W2801037557 @default.
- W2920121562 cites W2807750997 @default.
- W2920121562 cites W2809808428 @default.
- W2920121562 cites W2903841424 @default.
- W2920121562 cites W2919115771 @default.
- W2920121562 cites W2920954974 @default.
- W2920121562 cites W2921622930 @default.
- W2920121562 cites W2952448932 @default.
- W2920121562 cites W2963416336 @default.
- W2920121562 cites W2964115671 @default.
- W2920121562 cites W2964319207 @default.
- W2920121562 cites W2971713705 @default.
- W2920121562 cites W3022644913 @default.
- W2920121562 cites W3118608800 @default.
- W2920121562 cites W3203992401 @default.
- W2920121562 hasPublicationYear "2019" @default.
- W2920121562 type Work @default.
- W2920121562 sameAs 2920121562 @default.
- W2920121562 citedByCount "2" @default.
- W2920121562 countsByYear W29201215622019 @default.
- W2920121562 countsByYear W29201215622020 @default.
- W2920121562 crossrefType "posted-content" @default.
- W2920121562 hasAuthorship W2920121562A5016864001 @default.
- W2920121562 hasAuthorship W2920121562A5017755967 @default.
- W2920121562 hasAuthorship W2920121562A5089938000 @default.
- W2920121562 hasAuthorship W2920121562A5091408102 @default.
- W2920121562 hasConcept C108583219 @default.
- W2920121562 hasConcept C11413529 @default.
- W2920121562 hasConcept C121332964 @default.
- W2920121562 hasConcept C153258448 @default.
- W2920121562 hasConcept C154945302 @default.
- W2920121562 hasConcept C155032097 @default.
- W2920121562 hasConcept C163716315 @default.
- W2920121562 hasConcept C41008148 @default.
- W2920121562 hasConcept C45374587 @default.
- W2920121562 hasConcept C50644808 @default.
- W2920121562 hasConcept C56372850 @default.
- W2920121562 hasConcept C62520636 @default.
- W2920121562 hasConcept C81363708 @default.
- W2920121562 hasConceptScore W2920121562C108583219 @default.
- W2920121562 hasConceptScore W2920121562C11413529 @default.
- W2920121562 hasConceptScore W2920121562C121332964 @default.
- W2920121562 hasConceptScore W2920121562C153258448 @default.
- W2920121562 hasConceptScore W2920121562C154945302 @default.
- W2920121562 hasConceptScore W2920121562C155032097 @default.
- W2920121562 hasConceptScore W2920121562C163716315 @default.
- W2920121562 hasConceptScore W2920121562C41008148 @default.
- W2920121562 hasConceptScore W2920121562C45374587 @default.
- W2920121562 hasConceptScore W2920121562C50644808 @default.
- W2920121562 hasConceptScore W2920121562C56372850 @default.
- W2920121562 hasConceptScore W2920121562C62520636 @default.
- W2920121562 hasConceptScore W2920121562C81363708 @default.
- W2920121562 hasLocation W29201215621 @default.
- W2920121562 hasOpenAccess W2920121562 @default.
- W2920121562 hasPrimaryLocation W29201215621 @default.
- W2920121562 hasRelatedWork W1634767490 @default.
- W2920121562 hasRelatedWork W1956050456 @default.
- W2920121562 hasRelatedWork W2093503827 @default.
- W2920121562 hasRelatedWork W2618541429 @default.
- W2920121562 hasRelatedWork W2743024835 @default.
- W2920121562 hasRelatedWork W2768666611 @default.
- W2920121562 hasRelatedWork W2812009592 @default.
- W2920121562 hasRelatedWork W2885120244 @default.
- W2920121562 hasRelatedWork W2890311316 @default.
- W2920121562 hasRelatedWork W2892899012 @default.