Matches in SemOpenAlex for { <https://semopenalex.org/work/W4387076390> ?p ?o ?g. }
Showing items 1 to 63 of
63
with 100 items per page.
- W4387076390 abstract "Document-level Neural Machine Translation (DocNMT) has been proven crucial for handling discourse phenomena by introducing document-level context information. One of the most important directions is to input the whole document directly to the standard Transformer model. In this case, efficiency becomes a critical concern due to the quadratic complexity of the attention module. Existing studies either focus on the encoder part, which cannot be deployed on sequence-to-sequence generation tasks, e.g., Machine Translation (MT), or suffer from a significant performance drop. In this work, we keep the translation performance while gaining 20% speed up by introducing extra selection layer based on lightweight attention that selects a small portion of tokens to be attended. It takes advantage of the original attention to ensure performance and dimension reduction to accelerate inference. Experimental results show that our method could achieve up to 95% sparsity (only 5% tokens attended) approximately, and save 93% computation cost on the attention module compared with the original Transformer, while maintaining the performance." @default.
- W4387076390 created "2023-09-27" @default.
- W4387076390 creator A5006266198 @default.
- W4387076390 creator A5011760791 @default.
- W4387076390 creator A5025469065 @default.
- W4387076390 creator A5043256966 @default.
- W4387076390 creator A5046413480 @default.
- W4387076390 date "2023-09-25" @default.
- W4387076390 modified "2023-09-28" @default.
- W4387076390 title "Only 5% Attention Is All You Need: Efficient Long-range Document-level Neural Machine Translation" @default.
- W4387076390 doi "https://doi.org/10.48550/arxiv.2309.14174" @default.
- W4387076390 hasPublicationYear "2023" @default.
- W4387076390 type Work @default.
- W4387076390 citedByCount "0" @default.
- W4387076390 crossrefType "posted-content" @default.
- W4387076390 hasAuthorship W4387076390A5006266198 @default.
- W4387076390 hasAuthorship W4387076390A5011760791 @default.
- W4387076390 hasAuthorship W4387076390A5025469065 @default.
- W4387076390 hasAuthorship W4387076390A5043256966 @default.
- W4387076390 hasAuthorship W4387076390A5046413480 @default.
- W4387076390 hasBestOaLocation W43870763901 @default.
- W4387076390 hasConcept C111919701 @default.
- W4387076390 hasConcept C11413529 @default.
- W4387076390 hasConcept C118505674 @default.
- W4387076390 hasConcept C119599485 @default.
- W4387076390 hasConcept C119857082 @default.
- W4387076390 hasConcept C127413603 @default.
- W4387076390 hasConcept C154945302 @default.
- W4387076390 hasConcept C165801399 @default.
- W4387076390 hasConcept C203005215 @default.
- W4387076390 hasConcept C2776214188 @default.
- W4387076390 hasConcept C41008148 @default.
- W4387076390 hasConcept C45374587 @default.
- W4387076390 hasConcept C66322947 @default.
- W4387076390 hasConceptScore W4387076390C111919701 @default.
- W4387076390 hasConceptScore W4387076390C11413529 @default.
- W4387076390 hasConceptScore W4387076390C118505674 @default.
- W4387076390 hasConceptScore W4387076390C119599485 @default.
- W4387076390 hasConceptScore W4387076390C119857082 @default.
- W4387076390 hasConceptScore W4387076390C127413603 @default.
- W4387076390 hasConceptScore W4387076390C154945302 @default.
- W4387076390 hasConceptScore W4387076390C165801399 @default.
- W4387076390 hasConceptScore W4387076390C203005215 @default.
- W4387076390 hasConceptScore W4387076390C2776214188 @default.
- W4387076390 hasConceptScore W4387076390C41008148 @default.
- W4387076390 hasConceptScore W4387076390C45374587 @default.
- W4387076390 hasConceptScore W4387076390C66322947 @default.
- W4387076390 hasLocation W43870763901 @default.
- W4387076390 hasOpenAccess W4387076390 @default.
- W4387076390 hasPrimaryLocation W43870763901 @default.
- W4387076390 hasRelatedWork W2890964657 @default.
- W4387076390 hasRelatedWork W2903810591 @default.
- W4387076390 hasRelatedWork W2989276524 @default.
- W4387076390 hasRelatedWork W3006801027 @default.
- W4387076390 hasRelatedWork W3106504817 @default.
- W4387076390 hasRelatedWork W3110288483 @default.
- W4387076390 hasRelatedWork W4287864838 @default.
- W4387076390 hasRelatedWork W4289446079 @default.
- W4387076390 hasRelatedWork W4303874710 @default.
- W4387076390 hasRelatedWork W4313218062 @default.
- W4387076390 isParatext "false" @default.
- W4387076390 isRetracted "false" @default.
- W4387076390 workType "article" @default.