Matches in SemOpenAlex for { <https://semopenalex.org/work/W4288113199> ?p ?o ?g. }
Showing items 1 to 63 of
63
with 100 items per page.
- W4288113199 abstract "Neural models for NLP typically use large numbers of parameters to reach state-of-the-art performance, which can lead to excessive memory usage and increased runtime. We present a structure learning method for learning sparse, parameter-efficient NLP models. Our method applies group lasso to rational RNNs (Peng et al., 2018), a family of models that is closely connected to weighted finite-state automata (WFSAs). We take advantage of rational RNNs' natural grouping of the weights, so the group lasso penalty directly removes WFSA states, substantially reducing the number of parameters in the model. Our experiments on a number of sentiment analysis datasets, using both GloVe and BERT embeddings, show that our approach learns neural structures which have fewer parameters without sacrificing performance relative to parameter-rich baselines. Our method also highlights the interpretable properties of rational RNNs. We show that sparsifying such models makes them easier to visualize, and we present models that rely exclusively on as few as three WFSAs after pruning more than 90% of the weights. We publicly release our code." @default.
- W4288113199 created "2022-07-28" @default.
- W4288113199 creator A5007903277 @default.
- W4288113199 creator A5008013895 @default.
- W4288113199 creator A5069093595 @default.
- W4288113199 creator A5088517824 @default.
- W4288113199 date "2019-09-06" @default.
- W4288113199 modified "2023-09-26" @default.
- W4288113199 title "RNN Architecture Learning with Sparse Regularization" @default.
- W4288113199 doi "https://doi.org/10.48550/arxiv.1909.03011" @default.
- W4288113199 hasPublicationYear "2019" @default.
- W4288113199 type Work @default.
- W4288113199 citedByCount "0" @default.
- W4288113199 crossrefType "posted-content" @default.
- W4288113199 hasAuthorship W4288113199A5007903277 @default.
- W4288113199 hasAuthorship W4288113199A5008013895 @default.
- W4288113199 hasAuthorship W4288113199A5069093595 @default.
- W4288113199 hasAuthorship W4288113199A5088517824 @default.
- W4288113199 hasBestOaLocation W42881131991 @default.
- W4288113199 hasConcept C108010975 @default.
- W4288113199 hasConcept C119857082 @default.
- W4288113199 hasConcept C136764020 @default.
- W4288113199 hasConcept C147168706 @default.
- W4288113199 hasConcept C154945302 @default.
- W4288113199 hasConcept C177264268 @default.
- W4288113199 hasConcept C199360897 @default.
- W4288113199 hasConcept C2776135515 @default.
- W4288113199 hasConcept C2776760102 @default.
- W4288113199 hasConcept C37616216 @default.
- W4288113199 hasConcept C41008148 @default.
- W4288113199 hasConcept C50644808 @default.
- W4288113199 hasConcept C6557445 @default.
- W4288113199 hasConcept C86803240 @default.
- W4288113199 hasConceptScore W4288113199C108010975 @default.
- W4288113199 hasConceptScore W4288113199C119857082 @default.
- W4288113199 hasConceptScore W4288113199C136764020 @default.
- W4288113199 hasConceptScore W4288113199C147168706 @default.
- W4288113199 hasConceptScore W4288113199C154945302 @default.
- W4288113199 hasConceptScore W4288113199C177264268 @default.
- W4288113199 hasConceptScore W4288113199C199360897 @default.
- W4288113199 hasConceptScore W4288113199C2776135515 @default.
- W4288113199 hasConceptScore W4288113199C2776760102 @default.
- W4288113199 hasConceptScore W4288113199C37616216 @default.
- W4288113199 hasConceptScore W4288113199C41008148 @default.
- W4288113199 hasConceptScore W4288113199C50644808 @default.
- W4288113199 hasConceptScore W4288113199C6557445 @default.
- W4288113199 hasConceptScore W4288113199C86803240 @default.
- W4288113199 hasLocation W42881131991 @default.
- W4288113199 hasOpenAccess W4288113199 @default.
- W4288113199 hasPrimaryLocation W42881131991 @default.
- W4288113199 hasRelatedWork W2507865718 @default.
- W4288113199 hasRelatedWork W2767785892 @default.
- W4288113199 hasRelatedWork W2970848711 @default.
- W4288113199 hasRelatedWork W2972175708 @default.
- W4288113199 hasRelatedWork W2974521253 @default.
- W4288113199 hasRelatedWork W3174196512 @default.
- W4288113199 hasRelatedWork W3199608561 @default.
- W4288113199 hasRelatedWork W3213565677 @default.
- W4288113199 hasRelatedWork W4226184137 @default.
- W4288113199 hasRelatedWork W4283697347 @default.
- W4288113199 isParatext "false" @default.
- W4288113199 isRetracted "false" @default.
- W4288113199 workType "article" @default.