Matches in SemOpenAlex for { <https://semopenalex.org/work/W4385570909> ?p ?o ?g. }
Showing items 1 to 65 of
65
with 100 items per page.
- W4385570909 abstract "Transformer has become an important technique for natural language processing tasks with great success. However, it usually requires huge storage space and computational cost, making it difficult to be deployed on resource-constrained edge devices. To compress and accelerate Transformer, we propose LightFormer, which adopts a low-rank factorization initialized by SVD-based weight transfer and parameter sharing. The SVD-based weight transfer can effectively utilize the well-trained Transformer parameter knowledge to speed up the model convergence, and effectively alleviate the low-rank bottleneck problem combined with parameter sharing. We validate our method on machine translation, text summarization and text classification tasks. Experiments show that on IWSLT’14 De-En and WMT’14 En-De, LightFormer achieves similar performance to the baseline Transformer with 3.8 times and 1.8 times fewer parameters, and achieves 2.3 times speedup and 1.5 times speedup respectively, generally outperforming recent light-weight Transformers." @default.
- W4385570909 created "2023-08-05" @default.
- W4385570909 creator A5005091713 @default.
- W4385570909 creator A5022266260 @default.
- W4385570909 creator A5029360035 @default.
- W4385570909 creator A5047650196 @default.
- W4385570909 creator A5049787091 @default.
- W4385570909 date "2023-01-01" @default.
- W4385570909 modified "2023-09-24" @default.
- W4385570909 title "LightFormer: Light-weight Transformer Using SVD-based Weight Transfer and Parameter Sharing" @default.
- W4385570909 doi "https://doi.org/10.18653/v1/2023.findings-acl.656" @default.
- W4385570909 hasPublicationYear "2023" @default.
- W4385570909 type Work @default.
- W4385570909 citedByCount "0" @default.
- W4385570909 crossrefType "proceedings-article" @default.
- W4385570909 hasAuthorship W4385570909A5005091713 @default.
- W4385570909 hasAuthorship W4385570909A5022266260 @default.
- W4385570909 hasAuthorship W4385570909A5029360035 @default.
- W4385570909 hasAuthorship W4385570909A5047650196 @default.
- W4385570909 hasAuthorship W4385570909A5049787091 @default.
- W4385570909 hasBestOaLocation W43855709091 @default.
- W4385570909 hasConcept C11413529 @default.
- W4385570909 hasConcept C119599485 @default.
- W4385570909 hasConcept C127413603 @default.
- W4385570909 hasConcept C149635348 @default.
- W4385570909 hasConcept C154945302 @default.
- W4385570909 hasConcept C165801399 @default.
- W4385570909 hasConcept C170858558 @default.
- W4385570909 hasConcept C173608175 @default.
- W4385570909 hasConcept C203005215 @default.
- W4385570909 hasConcept C22789450 @default.
- W4385570909 hasConcept C2780513914 @default.
- W4385570909 hasConcept C41008148 @default.
- W4385570909 hasConcept C66322947 @default.
- W4385570909 hasConcept C68339613 @default.
- W4385570909 hasConceptScore W4385570909C11413529 @default.
- W4385570909 hasConceptScore W4385570909C119599485 @default.
- W4385570909 hasConceptScore W4385570909C127413603 @default.
- W4385570909 hasConceptScore W4385570909C149635348 @default.
- W4385570909 hasConceptScore W4385570909C154945302 @default.
- W4385570909 hasConceptScore W4385570909C165801399 @default.
- W4385570909 hasConceptScore W4385570909C170858558 @default.
- W4385570909 hasConceptScore W4385570909C173608175 @default.
- W4385570909 hasConceptScore W4385570909C203005215 @default.
- W4385570909 hasConceptScore W4385570909C22789450 @default.
- W4385570909 hasConceptScore W4385570909C2780513914 @default.
- W4385570909 hasConceptScore W4385570909C41008148 @default.
- W4385570909 hasConceptScore W4385570909C66322947 @default.
- W4385570909 hasConceptScore W4385570909C68339613 @default.
- W4385570909 hasLocation W43855709091 @default.
- W4385570909 hasOpenAccess W4385570909 @default.
- W4385570909 hasPrimaryLocation W43855709091 @default.
- W4385570909 hasRelatedWork W2351187795 @default.
- W4385570909 hasRelatedWork W2380641910 @default.
- W4385570909 hasRelatedWork W2891987081 @default.
- W4385570909 hasRelatedWork W2996766022 @default.
- W4385570909 hasRelatedWork W3007935045 @default.
- W4385570909 hasRelatedWork W3054488230 @default.
- W4385570909 hasRelatedWork W3083982194 @default.
- W4385570909 hasRelatedWork W3094342783 @default.
- W4385570909 hasRelatedWork W3155147984 @default.
- W4385570909 hasRelatedWork W4287669772 @default.
- W4385570909 isParatext "false" @default.
- W4385570909 isRetracted "false" @default.
- W4385570909 workType "article" @default.