Matches in SemOpenAlex for { <https://semopenalex.org/work/W2967472393> ?p ?o ?g. }
Showing items 1 to 85 of
85
with 100 items per page.
- W2967472393 abstract "Recent state-of-the-art natural language understanding models, such as BERT and XLNet, score a pair of sentences (A and B) using multiple cross-attention operations - a process in which each word in sentence A attends to all words in sentence B and vice versa. As a result, computing the similarity between a query sentence and a set of candidate sentences, requires the propagation of all query-candidate sentence-pairs throughout a stack of cross-attention layers. This exhaustive process becomes computationally prohibitive when the number of candidate sentences is large. In contrast, sentence embedding techniques learn a sentence-to-vector mapping and compute the similarity between the sentence vectors via simple elementary operations. In this paper, we introduce Distilled Sentence Embedding (DSE) - a model that is based on knowledge distillation from cross-attentive models, focusing on sentence-pair tasks. The outline of DSE is as follows: Given a cross-attentive teacher model (e.g. a fine-tuned BERT), we train a sentence embedding based student model to reconstruct the sentence-pair scores obtained by the teacher model. We empirically demonstrate the effectiveness of DSE on five GLUE sentence-pair tasks. DSE significantly outperforms several ELMO variants and other sentence embedding methods, while accelerating computation of the query-candidate sentence-pairs similarities by several orders of magnitude, with an average relative degradation of 4.6% compared to BERT. Furthermore, we show that DSE produces sentence embeddings that reach state-of-the-art performance on universal sentence representation benchmarks. Our code is made publicly available at this https URL." @default.
- W2967472393 created "2019-08-22" @default.
- W2967472393 creator A5030945372 @default.
- W2967472393 creator A5067773841 @default.
- W2967472393 creator A5077269072 @default.
- W2967472393 creator A5077903013 @default.
- W2967472393 creator A5083002103 @default.
- W2967472393 creator A5085243047 @default.
- W2967472393 date "2019-08-14" @default.
- W2967472393 modified "2023-09-23" @default.
- W2967472393 title "Scalable Attentive Sentence-Pair Modeling via Distilled Sentence Embedding" @default.
- W2967472393 cites W131533222 @default.
- W2967472393 cites W1486649854 @default.
- W2967472393 cites W1821462560 @default.
- W2967472393 cites W1840435438 @default.
- W2967472393 cites W2131462252 @default.
- W2967472393 cites W2153579005 @default.
- W2967472393 cites W2158899491 @default.
- W2967472393 cites W2170973209 @default.
- W2967472393 cites W2251939518 @default.
- W2967472393 cites W2761988601 @default.
- W2967472393 cites W2937297214 @default.
- W2967472393 cites W2950813464 @default.
- W2967472393 cites W2962739339 @default.
- W2967472393 cites W2963090765 @default.
- W2967472393 cites W2963310665 @default.
- W2967472393 cites W2963341956 @default.
- W2967472393 cites W2963403868 @default.
- W2967472393 cites W2963748441 @default.
- W2967472393 cites W2963846996 @default.
- W2967472393 cites W2963854351 @default.
- W2967472393 cites W2963918774 @default.
- W2967472393 cites W2964106018 @default.
- W2967472393 cites W2964121744 @default.
- W2967472393 cites W2964165804 @default.
- W2967472393 cites W2964341035 @default.
- W2967472393 cites W3104033643 @default.
- W2967472393 hasPublicationYear "2019" @default.
- W2967472393 type Work @default.
- W2967472393 sameAs 2967472393 @default.
- W2967472393 citedByCount "2" @default.
- W2967472393 countsByYear W29674723932020 @default.
- W2967472393 countsByYear W29674723932021 @default.
- W2967472393 crossrefType "posted-content" @default.
- W2967472393 hasAuthorship W2967472393A5030945372 @default.
- W2967472393 hasAuthorship W2967472393A5067773841 @default.
- W2967472393 hasAuthorship W2967472393A5077269072 @default.
- W2967472393 hasAuthorship W2967472393A5077903013 @default.
- W2967472393 hasAuthorship W2967472393A5083002103 @default.
- W2967472393 hasAuthorship W2967472393A5085243047 @default.
- W2967472393 hasConcept C154945302 @default.
- W2967472393 hasConcept C204321447 @default.
- W2967472393 hasConcept C2777530160 @default.
- W2967472393 hasConcept C41008148 @default.
- W2967472393 hasConcept C41608201 @default.
- W2967472393 hasConceptScore W2967472393C154945302 @default.
- W2967472393 hasConceptScore W2967472393C204321447 @default.
- W2967472393 hasConceptScore W2967472393C2777530160 @default.
- W2967472393 hasConceptScore W2967472393C41008148 @default.
- W2967472393 hasConceptScore W2967472393C41608201 @default.
- W2967472393 hasOpenAccess W2967472393 @default.
- W2967472393 hasRelatedWork W10934236 @default.
- W2967472393 hasRelatedWork W1968228387 @default.
- W2967472393 hasRelatedWork W2415204069 @default.
- W2967472393 hasRelatedWork W2575298912 @default.
- W2967472393 hasRelatedWork W2584932577 @default.
- W2967472393 hasRelatedWork W2760753016 @default.
- W2967472393 hasRelatedWork W2786028997 @default.
- W2967472393 hasRelatedWork W2889830342 @default.
- W2967472393 hasRelatedWork W2890032154 @default.
- W2967472393 hasRelatedWork W2997051142 @default.
- W2967472393 hasRelatedWork W3006690199 @default.
- W2967472393 hasRelatedWork W3021534166 @default.
- W2967472393 hasRelatedWork W3102335248 @default.
- W2967472393 hasRelatedWork W3121954283 @default.
- W2967472393 hasRelatedWork W3126835783 @default.
- W2967472393 hasRelatedWork W3134942116 @default.
- W2967472393 hasRelatedWork W3152569586 @default.
- W2967472393 hasRelatedWork W3202009528 @default.
- W2967472393 hasRelatedWork W3213956755 @default.
- W2967472393 hasRelatedWork W2740932779 @default.
- W2967472393 isParatext "false" @default.
- W2967472393 isRetracted "false" @default.
- W2967472393 magId "2967472393" @default.
- W2967472393 workType "article" @default.