Matches in SemOpenAlex for { <https://semopenalex.org/work/W2899062006> ?p ?o ?g. }
Showing items 1 to 81 of
81
with 100 items per page.
- W2899062006 abstract "This paper carries out an empirical analysis of various dropout techniques for language modelling, such as Bernoulli dropout, Gaussian dropout, Curriculum Dropout, Variational Dropout and Concrete Dropout. Moreover, we propose an extension of variational dropout to concrete dropout and curriculum dropout with varying schedules. We find these extensions to perform well when compared to standard dropout approaches, particularly variational curriculum dropout with a linear schedule. Largest performance increases are made when applying dropout on the decoder layer. Lastly, we analyze where most of the errors occur at test time as a post-analysis step to determine if the well-known problem of compounding errors is apparent and to what end do the proposed methods mitigate this issue for each dataset. We report results on a 2-hidden layer LSTM, GRU and Highway network with embedding dropout, dropout on the gated hidden layers and the output projection layer for each model. We report our results on Penn-TreeBank and WikiText-2 word-level language modelling datasets, where the former reduces the long-tail distribution through preprocessing and one which preserves rare words in the training and test set." @default.
- W2899062006 created "2018-11-09" @default.
- W2899062006 creator A5073503574 @default.
- W2899062006 creator A5083742715 @default.
- W2899062006 date "2018-11-02" @default.
- W2899062006 modified "2023-10-17" @default.
- W2899062006 title "Analysing Dropout and Compounding Errors in Neural Language Models." @default.
- W2899062006 cites W1591801644 @default.
- W2899062006 cites W1904365287 @default.
- W2899062006 cites W2136836265 @default.
- W2899062006 cites W2487501366 @default.
- W2899062006 cites W2508728158 @default.
- W2899062006 cites W2518108298 @default.
- W2899062006 cites W2525332836 @default.
- W2899062006 cites W2549416390 @default.
- W2899062006 cites W2616619952 @default.
- W2899062006 cites W2732547613 @default.
- W2899062006 cites W2743945814 @default.
- W2899062006 cites W2767630563 @default.
- W2899062006 cites W2781596748 @default.
- W2899062006 cites W2792376130 @default.
- W2899062006 cites W2891543607 @default.
- W2899062006 cites W2951595529 @default.
- W2899062006 cites W2951672049 @default.
- W2899062006 cites W2952088488 @default.
- W2899062006 cites W2952436057 @default.
- W2899062006 cites W2952840881 @default.
- W2899062006 cites W2963266340 @default.
- W2899062006 cites W2963748792 @default.
- W2899062006 cites W2964054038 @default.
- W2899062006 cites W3037590790 @default.
- W2899062006 hasPublicationYear "2018" @default.
- W2899062006 type Work @default.
- W2899062006 sameAs 2899062006 @default.
- W2899062006 citedByCount "0" @default.
- W2899062006 crossrefType "posted-content" @default.
- W2899062006 hasAuthorship W2899062006A5073503574 @default.
- W2899062006 hasAuthorship W2899062006A5083742715 @default.
- W2899062006 hasConcept C119857082 @default.
- W2899062006 hasConcept C121332964 @default.
- W2899062006 hasConcept C154945302 @default.
- W2899062006 hasConcept C163716315 @default.
- W2899062006 hasConcept C2776145597 @default.
- W2899062006 hasConcept C41008148 @default.
- W2899062006 hasConcept C50644808 @default.
- W2899062006 hasConcept C62520636 @default.
- W2899062006 hasConceptScore W2899062006C119857082 @default.
- W2899062006 hasConceptScore W2899062006C121332964 @default.
- W2899062006 hasConceptScore W2899062006C154945302 @default.
- W2899062006 hasConceptScore W2899062006C163716315 @default.
- W2899062006 hasConceptScore W2899062006C2776145597 @default.
- W2899062006 hasConceptScore W2899062006C41008148 @default.
- W2899062006 hasConceptScore W2899062006C50644808 @default.
- W2899062006 hasConceptScore W2899062006C62520636 @default.
- W2899062006 hasLocation W28990620061 @default.
- W2899062006 hasOpenAccess W2899062006 @default.
- W2899062006 hasPrimaryLocation W28990620061 @default.
- W2899062006 hasRelatedWork W146576602 @default.
- W2899062006 hasRelatedWork W2589744741 @default.
- W2899062006 hasRelatedWork W2716974933 @default.
- W2899062006 hasRelatedWork W2783846682 @default.
- W2899062006 hasRelatedWork W2792147139 @default.
- W2899062006 hasRelatedWork W2896866209 @default.
- W2899062006 hasRelatedWork W2917759141 @default.
- W2899062006 hasRelatedWork W2948397587 @default.
- W2899062006 hasRelatedWork W2963606129 @default.
- W2899062006 hasRelatedWork W2969418762 @default.
- W2899062006 hasRelatedWork W3009236732 @default.
- W2899062006 hasRelatedWork W3010624034 @default.
- W2899062006 hasRelatedWork W3041928889 @default.
- W2899062006 hasRelatedWork W3119641295 @default.
- W2899062006 hasRelatedWork W3135699790 @default.
- W2899062006 hasRelatedWork W3136808611 @default.
- W2899062006 hasRelatedWork W3166886836 @default.
- W2899062006 hasRelatedWork W3175302149 @default.
- W2899062006 hasRelatedWork W3207251398 @default.
- W2899062006 hasRelatedWork W3208281152 @default.
- W2899062006 isParatext "false" @default.
- W2899062006 isRetracted "false" @default.
- W2899062006 magId "2899062006" @default.
- W2899062006 workType "article" @default.