Matches in SemOpenAlex for { <https://semopenalex.org/work/W4380551413> ?p ?o ?g. }
Showing items 1 to 73 of
73
with 100 items per page.
- W4380551413 abstract "Non-autoregressive approaches aim to improve the inference speed of translation models, particularly those that generate output in a one-pass forward manner. However, these approaches often suffer from a significant drop in translation quality compared to autoregressive models. This paper introduces a series of innovative techniques to enhance the translation quality of Non-Autoregressive Translation (NAT) models while maintaining a substantial acceleration in inference speed. We propose fine-tuning Pretrained Multilingual Language Models (PMLMs) with the CTC loss to train NAT models effectively. Furthermore, we adopt the MASK insertion scheme for up-sampling instead of token duplication, and we present an embedding distillation method to further enhance performance. In our experiments, our model outperforms the baseline autoregressive model (Transformer textit{base}) on multiple datasets, including WMT'14 DE$leftrightarrow$EN, WMT'16 RO$leftrightarrow$EN, and IWSLT'14 DE$leftrightarrow$EN. Notably, our model achieves better performance than the baseline autoregressive model on the IWSLT'14 En$leftrightarrow$De and WMT'16 En$leftrightarrow$Ro datasets, even without using distillation data during training. It is worth highlighting that on the IWSLT'14 DE$rightarrow$EN dataset, our model achieves an impressive BLEU score of 39.59, setting a new state-of-the-art performance. Additionally, our model exhibits a remarkable speed improvement of 16.35 times compared to the autoregressive model." @default.
- W4380551413 created "2023-06-14" @default.
- W4380551413 creator A5040508737 @default.
- W4380551413 creator A5054647497 @default.
- W4380551413 creator A5056141038 @default.
- W4380551413 date "2023-06-10" @default.
- W4380551413 modified "2023-10-01" @default.
- W4380551413 title "Improving Non-autoregressive Translation Quality with Pretrained Language Model, Embedding Distillation and Upsampling Strategy for CTC" @default.
- W4380551413 doi "https://doi.org/10.48550/arxiv.2306.06345" @default.
- W4380551413 hasPublicationYear "2023" @default.
- W4380551413 type Work @default.
- W4380551413 citedByCount "0" @default.
- W4380551413 crossrefType "posted-content" @default.
- W4380551413 hasAuthorship W4380551413A5040508737 @default.
- W4380551413 hasAuthorship W4380551413A5054647497 @default.
- W4380551413 hasAuthorship W4380551413A5056141038 @default.
- W4380551413 hasBestOaLocation W43805514131 @default.
- W4380551413 hasConcept C104317684 @default.
- W4380551413 hasConcept C105580179 @default.
- W4380551413 hasConcept C11413529 @default.
- W4380551413 hasConcept C121332964 @default.
- W4380551413 hasConcept C137293760 @default.
- W4380551413 hasConcept C149364088 @default.
- W4380551413 hasConcept C149782125 @default.
- W4380551413 hasConcept C154945302 @default.
- W4380551413 hasConcept C159877910 @default.
- W4380551413 hasConcept C165801399 @default.
- W4380551413 hasConcept C185592680 @default.
- W4380551413 hasConcept C203005215 @default.
- W4380551413 hasConcept C2776214188 @default.
- W4380551413 hasConcept C33923547 @default.
- W4380551413 hasConcept C41008148 @default.
- W4380551413 hasConcept C41608201 @default.
- W4380551413 hasConcept C55493867 @default.
- W4380551413 hasConcept C622187 @default.
- W4380551413 hasConcept C62520636 @default.
- W4380551413 hasConcept C66322947 @default.
- W4380551413 hasConceptScore W4380551413C104317684 @default.
- W4380551413 hasConceptScore W4380551413C105580179 @default.
- W4380551413 hasConceptScore W4380551413C11413529 @default.
- W4380551413 hasConceptScore W4380551413C121332964 @default.
- W4380551413 hasConceptScore W4380551413C137293760 @default.
- W4380551413 hasConceptScore W4380551413C149364088 @default.
- W4380551413 hasConceptScore W4380551413C149782125 @default.
- W4380551413 hasConceptScore W4380551413C154945302 @default.
- W4380551413 hasConceptScore W4380551413C159877910 @default.
- W4380551413 hasConceptScore W4380551413C165801399 @default.
- W4380551413 hasConceptScore W4380551413C185592680 @default.
- W4380551413 hasConceptScore W4380551413C203005215 @default.
- W4380551413 hasConceptScore W4380551413C2776214188 @default.
- W4380551413 hasConceptScore W4380551413C33923547 @default.
- W4380551413 hasConceptScore W4380551413C41008148 @default.
- W4380551413 hasConceptScore W4380551413C41608201 @default.
- W4380551413 hasConceptScore W4380551413C55493867 @default.
- W4380551413 hasConceptScore W4380551413C622187 @default.
- W4380551413 hasConceptScore W4380551413C62520636 @default.
- W4380551413 hasConceptScore W4380551413C66322947 @default.
- W4380551413 hasLocation W43805514131 @default.
- W4380551413 hasOpenAccess W4380551413 @default.
- W4380551413 hasPrimaryLocation W43805514131 @default.
- W4380551413 hasRelatedWork W2626778328 @default.
- W4380551413 hasRelatedWork W2996854111 @default.
- W4380551413 hasRelatedWork W3155823939 @default.
- W4380551413 hasRelatedWork W3200578235 @default.
- W4380551413 hasRelatedWork W3206689743 @default.
- W4380551413 hasRelatedWork W3212566403 @default.
- W4380551413 hasRelatedWork W4303874710 @default.
- W4380551413 hasRelatedWork W4317438660 @default.
- W4380551413 hasRelatedWork W4366082296 @default.
- W4380551413 hasRelatedWork W4385245566 @default.
- W4380551413 isParatext "false" @default.
- W4380551413 isRetracted "false" @default.
- W4380551413 workType "article" @default.