FRINK Linked Data Fragments server

Matches in SemOpenAlex for { <https://semopenalex.org/work/W3173414161> ?p ?o ?g. }

• W3173414161 abstract "Recurrent Neural Networks (RNNs), more specifically their Long Short-Term Memory (LSTM) variants, have been widely used as a deep learning tool for tackling sequence-based learning tasks in text and speech. Training of such LSTM applications is computationally intensive due to the recurrent nature of hidden state computation that repeats for each time step. While sparsity in Deep Neural Nets has been widely seen as an opportunity for reducing computation time in both training and inference phases, the usage of non-ReLU activation in LSTM RNNs renders the opportunities for such dynamic sparsity associated with neuron activation and gradient values to be limited or non-existent. In this work, we identify dropout induced sparsity for LSTMs as a suitable mode of computation reduction. Dropout is a widely used regularization mechanism, which randomly drops computed neuron values during each iteration of training. We propose to structure dropout patterns, by dropping out the same set of physical neurons within a batch, resulting in column (row) level hidden state sparsity, which are well amenable to computation reduction at run-time in general-purpose SIMD hardware as well as systolic arrays. We conduct our experiments for three representative NLP tasks: language modelling on the PTB dataset, OpenNMT based machine translation using the IWSLT De-En and En-Vi datasets, and named entity recognition sequence labelling using the CoNLL-2003 shared task. We demonstrate that our proposed approach can be used to translate dropout-based computation reduction into reduced training time, with improvement ranging from 1.23x to 1.64x, without sacrificing the target metric." @default.
• W3173414161 created "2021-07-05" @default.
• W3173414161 creator A5007116603 @default.
• W3173414161 creator A5054027488 @default.
• W3173414161 creator A5058385222 @default.
• W3173414161 creator A5065037360 @default.
• W3173414161 creator A5067815274 @default.
• W3173414161 creator A5071846463 @default.
• W3173414161 date "2021-06-22" @default.
• W3173414161 modified "2023-09-27" @default.
• W3173414161 title "Structured in Space, Randomized in Time: Leveraging Dropout in RNNs for Efficient Training." @default.
• W3173414161 cites W1591801644 @default.
• W3173414161 cites W1632114991 @default.
• W3173414161 cites W1665214252 @default.
• W3173414161 cites W1902237438 @default.
• W3173414161 cites W1904365287 @default.
• W3173414161 cites W1922655562 @default.
• W3173414161 cites W1999965501 @default.
• W3173414161 cites W2064675550 @default.
• W3173414161 cites W2143612262 @default.
• W3173414161 cites W2150355110 @default.
• W3173414161 cites W2292443655 @default.
• W3173414161 cites W2300605907 @default.
• W3173414161 cites W2409027918 @default.
• W3173414161 cites W2513419314 @default.
• W3173414161 cites W2608554408 @default.
• W3173414161 cites W2619096655 @default.
• W3173414161 cites W2657126969 @default.
• W3173414161 cites W2743945814 @default.
• W3173414161 cites W2754526845 @default.
• W3173414161 cites W2767785892 @default.
• W3173414161 cites W2788810909 @default.
• W3173414161 cites W2951595529 @default.
• W3173414161 cites W2952352518 @default.
• W3173414161 cites W2962902328 @default.
• W3173414161 cites W2963212250 @default.
• W3173414161 cites W2963674932 @default.
• W3173414161 cites W2963991999 @default.
• W3173414161 cites W2964059111 @default.
• W3173414161 cites W2964299589 @default.
• W3173414161 cites W2964325005 @default.
• W3173414161 cites W2984140583 @default.
• W3173414161 cites W2991232789 @default.
• W3173414161 cites W4919037 @default.
• W3173414161 hasPublicationYear "2021" @default.
• W3173414161 type Work @default.
• W3173414161 sameAs 3173414161 @default.
• W3173414161 citedByCount "0" @default.
• W3173414161 crossrefType "posted-content" @default.
• W3173414161 hasAuthorship W3173414161A5007116603 @default.
• W3173414161 hasAuthorship W3173414161A5054027488 @default.
• W3173414161 hasAuthorship W3173414161A5058385222 @default.
• W3173414161 hasAuthorship W3173414161A5065037360 @default.
• W3173414161 hasAuthorship W3173414161A5067815274 @default.
• W3173414161 hasAuthorship W3173414161A5071846463 @default.
• W3173414161 hasConcept C111335779 @default.
• W3173414161 hasConcept C11413529 @default.
• W3173414161 hasConcept C119857082 @default.
• W3173414161 hasConcept C137293760 @default.
• W3173414161 hasConcept C147168706 @default.
• W3173414161 hasConcept C154945302 @default.
• W3173414161 hasConcept C173608175 @default.
• W3173414161 hasConcept C203005215 @default.
• W3173414161 hasConcept C2524010 @default.
• W3173414161 hasConcept C2776135515 @default.
• W3173414161 hasConcept C2776145597 @default.
• W3173414161 hasConcept C2776214188 @default.
• W3173414161 hasConcept C2778112365 @default.
• W3173414161 hasConcept C28490314 @default.
• W3173414161 hasConcept C33923547 @default.
• W3173414161 hasConcept C41008148 @default.
• W3173414161 hasConcept C45374587 @default.
• W3173414161 hasConcept C50644808 @default.
• W3173414161 hasConcept C54355233 @default.
• W3173414161 hasConcept C68339613 @default.
• W3173414161 hasConcept C86803240 @default.
• W3173414161 hasConceptScore W3173414161C111335779 @default.
• W3173414161 hasConceptScore W3173414161C11413529 @default.
• W3173414161 hasConceptScore W3173414161C119857082 @default.
• W3173414161 hasConceptScore W3173414161C137293760 @default.
• W3173414161 hasConceptScore W3173414161C147168706 @default.
• W3173414161 hasConceptScore W3173414161C154945302 @default.
• W3173414161 hasConceptScore W3173414161C173608175 @default.
• W3173414161 hasConceptScore W3173414161C203005215 @default.
• W3173414161 hasConceptScore W3173414161C2524010 @default.
• W3173414161 hasConceptScore W3173414161C2776135515 @default.
• W3173414161 hasConceptScore W3173414161C2776145597 @default.
• W3173414161 hasConceptScore W3173414161C2776214188 @default.
• W3173414161 hasConceptScore W3173414161C2778112365 @default.
• W3173414161 hasConceptScore W3173414161C28490314 @default.
• W3173414161 hasConceptScore W3173414161C33923547 @default.
• W3173414161 hasConceptScore W3173414161C41008148 @default.
• W3173414161 hasConceptScore W3173414161C45374587 @default.
• W3173414161 hasConceptScore W3173414161C50644808 @default.
• W3173414161 hasConceptScore W3173414161C54355233 @default.
• W3173414161 hasConceptScore W3173414161C68339613 @default.
• W3173414161 hasConceptScore W3173414161C86803240 @default.
• W3173414161 hasLocation W31734141611 @default.
• W3173414161 hasOpenAccess W3173414161 @default.
• W3173414161 hasPrimaryLocation W31734141611 @default.

• next