Matches in SemOpenAlex for { <https://semopenalex.org/work/W4385571044> ?p ?o ?g. }
Showing items 1 to 93 of
93
with 100 items per page.
- W4385571044 abstract "The computational benefits of iterative non-autoregressive transformers decrease as the number of decoding steps increases.As a remedy, we introduce Distill Multiple Steps (DiMS), a simple yet effective distillation technique to decrease the number of required steps to reach a certain translation quality.The distilled model enjoys the computational benefits of early iterations while preserving the enhancements from several iterative steps.DiMS relies on two models namely student and teacher.The student is optimized to predict the output of the teacher after multiple decoding steps while the teacher follows the student via a slow-moving average.The moving average keeps the teacher’s knowledge updated and enhances the quality of the labels provided by the teacher.During inference, the student is used for translation and no additional computation is added.We verify the effectiveness of DiMS on various models obtaining 7.8 and 12.9 BLEU points improvements in single-step translation accuracy on distilled and raw versions of WMT’14 De-En.Full code for this work is available here: https://github.com/layer6ai-labs/DiMS" @default.
- W4385571044 created "2023-08-05" @default.
- W4385571044 creator A5027341076 @default.
- W4385571044 creator A5046086284 @default.
- W4385571044 creator A5058024386 @default.
- W4385571044 creator A5067693060 @default.
- W4385571044 date "2023-01-01" @default.
- W4385571044 modified "2023-09-24" @default.
- W4385571044 title "DiMS: Distilling Multiple Steps of Iterative Non-Autoregressive Transformers for Machine Translation" @default.
- W4385571044 doi "https://doi.org/10.18653/v1/2023.findings-acl.542" @default.
- W4385571044 hasPublicationYear "2023" @default.
- W4385571044 type Work @default.
- W4385571044 citedByCount "0" @default.
- W4385571044 crossrefType "proceedings-article" @default.
- W4385571044 hasAuthorship W4385571044A5027341076 @default.
- W4385571044 hasAuthorship W4385571044A5046086284 @default.
- W4385571044 hasAuthorship W4385571044A5058024386 @default.
- W4385571044 hasAuthorship W4385571044A5067693060 @default.
- W4385571044 hasBestOaLocation W43855710441 @default.
- W4385571044 hasConcept C104317684 @default.
- W4385571044 hasConcept C105580179 @default.
- W4385571044 hasConcept C105795698 @default.
- W4385571044 hasConcept C111472728 @default.
- W4385571044 hasConcept C113775141 @default.
- W4385571044 hasConcept C11413529 @default.
- W4385571044 hasConcept C119857082 @default.
- W4385571044 hasConcept C121332964 @default.
- W4385571044 hasConcept C138885662 @default.
- W4385571044 hasConcept C149364088 @default.
- W4385571044 hasConcept C154945302 @default.
- W4385571044 hasConcept C159694833 @default.
- W4385571044 hasConcept C159877910 @default.
- W4385571044 hasConcept C165801399 @default.
- W4385571044 hasConcept C177264268 @default.
- W4385571044 hasConcept C185592680 @default.
- W4385571044 hasConcept C199360897 @default.
- W4385571044 hasConcept C203005215 @default.
- W4385571044 hasConcept C2776214188 @default.
- W4385571044 hasConcept C2776760102 @default.
- W4385571044 hasConcept C2779982483 @default.
- W4385571044 hasConcept C2780586882 @default.
- W4385571044 hasConcept C33923547 @default.
- W4385571044 hasConcept C41008148 @default.
- W4385571044 hasConcept C45374587 @default.
- W4385571044 hasConcept C55493867 @default.
- W4385571044 hasConcept C57273362 @default.
- W4385571044 hasConcept C62520636 @default.
- W4385571044 hasConcept C66322947 @default.
- W4385571044 hasConceptScore W4385571044C104317684 @default.
- W4385571044 hasConceptScore W4385571044C105580179 @default.
- W4385571044 hasConceptScore W4385571044C105795698 @default.
- W4385571044 hasConceptScore W4385571044C111472728 @default.
- W4385571044 hasConceptScore W4385571044C113775141 @default.
- W4385571044 hasConceptScore W4385571044C11413529 @default.
- W4385571044 hasConceptScore W4385571044C119857082 @default.
- W4385571044 hasConceptScore W4385571044C121332964 @default.
- W4385571044 hasConceptScore W4385571044C138885662 @default.
- W4385571044 hasConceptScore W4385571044C149364088 @default.
- W4385571044 hasConceptScore W4385571044C154945302 @default.
- W4385571044 hasConceptScore W4385571044C159694833 @default.
- W4385571044 hasConceptScore W4385571044C159877910 @default.
- W4385571044 hasConceptScore W4385571044C165801399 @default.
- W4385571044 hasConceptScore W4385571044C177264268 @default.
- W4385571044 hasConceptScore W4385571044C185592680 @default.
- W4385571044 hasConceptScore W4385571044C199360897 @default.
- W4385571044 hasConceptScore W4385571044C203005215 @default.
- W4385571044 hasConceptScore W4385571044C2776214188 @default.
- W4385571044 hasConceptScore W4385571044C2776760102 @default.
- W4385571044 hasConceptScore W4385571044C2779982483 @default.
- W4385571044 hasConceptScore W4385571044C2780586882 @default.
- W4385571044 hasConceptScore W4385571044C33923547 @default.
- W4385571044 hasConceptScore W4385571044C41008148 @default.
- W4385571044 hasConceptScore W4385571044C45374587 @default.
- W4385571044 hasConceptScore W4385571044C55493867 @default.
- W4385571044 hasConceptScore W4385571044C57273362 @default.
- W4385571044 hasConceptScore W4385571044C62520636 @default.
- W4385571044 hasConceptScore W4385571044C66322947 @default.
- W4385571044 hasLocation W43855710441 @default.
- W4385571044 hasOpenAccess W4385571044 @default.
- W4385571044 hasPrimaryLocation W43855710441 @default.
- W4385571044 hasRelatedWork W2917049097 @default.
- W4385571044 hasRelatedWork W2972767494 @default.
- W4385571044 hasRelatedWork W2988536374 @default.
- W4385571044 hasRelatedWork W3054488230 @default.
- W4385571044 hasRelatedWork W3116857254 @default.
- W4385571044 hasRelatedWork W3185923046 @default.
- W4385571044 hasRelatedWork W3206689743 @default.
- W4385571044 hasRelatedWork W3214236736 @default.
- W4385571044 hasRelatedWork W4303874710 @default.
- W4385571044 hasRelatedWork W4317438660 @default.
- W4385571044 isParatext "false" @default.
- W4385571044 isRetracted "false" @default.
- W4385571044 workType "article" @default.