Matches in SemOpenAlex for { <https://semopenalex.org/work/W2962819663> ?p ?o ?g. }
Showing items 1 to 99 of
99
with 100 items per page.
- W2962819663 abstract "Abstract: We propose BlackOut, an approximation algorithm to efficiently train massive recurrent neural network language models (RNNLMs) with million word vocabularies. BlackOut is motivated by using a discriminative loss, and we describe a new sampling strategy which significantly reduces computation while improving stability, sample efficiency, and rate of convergence. One way to understand BlackOut is to view it as an extension of the DropOut strategy to the output layer, wherein we use a discriminative training loss and a weighted sampling scheme. We also establish close connections between BlackOut, importance sampling, and noise contrastive estimation (NCE). Our experiments, on the recently released one billion word language modeling benchmark, demonstrate scalability and accuracy of BlackOut; we outperform the state-of-the art, and achieve the lowest perplexity scores on this dataset. Moreover, unlike other established methods which typically require GPUs or CPU clusters, we show that a carefully implemented version of BlackOut requires only 1-10 days on a single machine to train a RNNLM with a million word vocabulary and billions of parameters on one billion words. Although we describe BlackOut in the context of RNNLM training, it can be used to any networks with large softmax output layers." @default.
- W2962819663 created "2019-07-30" @default.
- W2962819663 creator A5013623933 @default.
- W2962819663 creator A5016936288 @default.
- W2962819663 creator A5032238070 @default.
- W2962819663 creator A5036338045 @default.
- W2962819663 creator A5048688625 @default.
- W2962819663 date "2016-01-01" @default.
- W2962819663 modified "2023-09-28" @default.
- W2962819663 title "BlackOut: Speeding up Recurrent Neural Network Language Models With Very Large Vocabularies" @default.
- W2962819663 hasPublicationYear "2016" @default.
- W2962819663 type Work @default.
- W2962819663 sameAs 2962819663 @default.
- W2962819663 citedByCount "34" @default.
- W2962819663 countsByYear W29628196632015 @default.
- W2962819663 countsByYear W29628196632016 @default.
- W2962819663 countsByYear W29628196632017 @default.
- W2962819663 countsByYear W29628196632018 @default.
- W2962819663 countsByYear W29628196632019 @default.
- W2962819663 countsByYear W29628196632020 @default.
- W2962819663 crossrefType "proceedings-article" @default.
- W2962819663 hasAuthorship W2962819663A5013623933 @default.
- W2962819663 hasAuthorship W2962819663A5016936288 @default.
- W2962819663 hasAuthorship W2962819663A5032238070 @default.
- W2962819663 hasAuthorship W2962819663A5036338045 @default.
- W2962819663 hasAuthorship W2962819663A5048688625 @default.
- W2962819663 hasConcept C100279451 @default.
- W2962819663 hasConcept C119857082 @default.
- W2962819663 hasConcept C121332964 @default.
- W2962819663 hasConcept C13280743 @default.
- W2962819663 hasConcept C137293760 @default.
- W2962819663 hasConcept C138885662 @default.
- W2962819663 hasConcept C154945302 @default.
- W2962819663 hasConcept C163258240 @default.
- W2962819663 hasConcept C185798385 @default.
- W2962819663 hasConcept C188441871 @default.
- W2962819663 hasConcept C205649164 @default.
- W2962819663 hasConcept C2777601683 @default.
- W2962819663 hasConcept C2777693866 @default.
- W2962819663 hasConcept C41008148 @default.
- W2962819663 hasConcept C41895202 @default.
- W2962819663 hasConcept C48044578 @default.
- W2962819663 hasConcept C50644808 @default.
- W2962819663 hasConcept C62520636 @default.
- W2962819663 hasConcept C77088390 @default.
- W2962819663 hasConcept C774472 @default.
- W2962819663 hasConcept C89227174 @default.
- W2962819663 hasConcept C90805587 @default.
- W2962819663 hasConcept C97931131 @default.
- W2962819663 hasConceptScore W2962819663C100279451 @default.
- W2962819663 hasConceptScore W2962819663C119857082 @default.
- W2962819663 hasConceptScore W2962819663C121332964 @default.
- W2962819663 hasConceptScore W2962819663C13280743 @default.
- W2962819663 hasConceptScore W2962819663C137293760 @default.
- W2962819663 hasConceptScore W2962819663C138885662 @default.
- W2962819663 hasConceptScore W2962819663C154945302 @default.
- W2962819663 hasConceptScore W2962819663C163258240 @default.
- W2962819663 hasConceptScore W2962819663C185798385 @default.
- W2962819663 hasConceptScore W2962819663C188441871 @default.
- W2962819663 hasConceptScore W2962819663C205649164 @default.
- W2962819663 hasConceptScore W2962819663C2777601683 @default.
- W2962819663 hasConceptScore W2962819663C2777693866 @default.
- W2962819663 hasConceptScore W2962819663C41008148 @default.
- W2962819663 hasConceptScore W2962819663C41895202 @default.
- W2962819663 hasConceptScore W2962819663C48044578 @default.
- W2962819663 hasConceptScore W2962819663C50644808 @default.
- W2962819663 hasConceptScore W2962819663C62520636 @default.
- W2962819663 hasConceptScore W2962819663C77088390 @default.
- W2962819663 hasConceptScore W2962819663C774472 @default.
- W2962819663 hasConceptScore W2962819663C89227174 @default.
- W2962819663 hasConceptScore W2962819663C90805587 @default.
- W2962819663 hasConceptScore W2962819663C97931131 @default.
- W2962819663 hasLocation W29628196631 @default.
- W2962819663 hasOpenAccess W2962819663 @default.
- W2962819663 hasPrimaryLocation W29628196631 @default.
- W2962819663 hasRelatedWork W1558797106 @default.
- W2962819663 hasRelatedWork W179875071 @default.
- W2962819663 hasRelatedWork W1902237438 @default.
- W2962819663 hasRelatedWork W2064675550 @default.
- W2962819663 hasRelatedWork W2100664567 @default.
- W2962819663 hasRelatedWork W2101105183 @default.
- W2962819663 hasRelatedWork W2130942839 @default.
- W2962819663 hasRelatedWork W2131462252 @default.
- W2962819663 hasRelatedWork W2138204974 @default.
- W2962819663 hasRelatedWork W2152790380 @default.
- W2962819663 hasRelatedWork W2153508793 @default.
- W2962819663 hasRelatedWork W2153579005 @default.
- W2962819663 hasRelatedWork W2175585630 @default.
- W2962819663 hasRelatedWork W2259472270 @default.
- W2962819663 hasRelatedWork W2463033603 @default.
- W2962819663 hasRelatedWork W2962784628 @default.
- W2962819663 hasRelatedWork W2963932686 @default.
- W2962819663 hasRelatedWork W2964308564 @default.
- W2962819663 hasRelatedWork W36903255 @default.
- W2962819663 hasRelatedWork W581956982 @default.
- W2962819663 isParatext "false" @default.
- W2962819663 isRetracted "false" @default.
- W2962819663 magId "2962819663" @default.
- W2962819663 workType "article" @default.