Matches in SemOpenAlex for { <https://semopenalex.org/work/W4367016469> ?p ?o ?g. }
Showing items 1 to 63 of
63
with 100 items per page.
- W4367016469 abstract "Recently, the self-attention mechanism (Transformer) has shown its advantages in various natural language processing (NLP) tasks. Since positional information is crucial to NLP tasks, the positional encoding has become a critical factor in improving the performance of the Transformer. In this paper, we present a simple but effective complex-valued relative positional encoding (CRPE) method. Specifically, we map the query and key vectors to the complex domain based on their positions. Hence, the attention weights will directly contain the relative positional information by the dot product between the complex-valued query and key vectors. To demonstrate the effectiveness of our method, we use four typical NLP tasks: named entity recognition, text classification, machine translation, and language modeling. The datasets of these tasks comprise texts of varying lengths. In the experiments, our method outperforms the baseline positional encodings across all datasets. The results show that our method is more effective for long and short texts while containing fewer parameters." @default.
- W4367016469 created "2023-04-27" @default.
- W4367016469 creator A5002627939 @default.
- W4367016469 creator A5038264616 @default.
- W4367016469 date "2023-02-24" @default.
- W4367016469 modified "2023-10-17" @default.
- W4367016469 title "Complex-Valued Relative Positional Encodings for Transformer" @default.
- W4367016469 cites W1902237438 @default.
- W4367016469 cites W2014902591 @default.
- W4367016469 cites W2064675550 @default.
- W4367016469 cites W2070246124 @default.
- W4367016469 cites W2114524997 @default.
- W4367016469 cites W2160660844 @default.
- W4367016469 cites W2163455955 @default.
- W4367016469 cites W2250539671 @default.
- W4367016469 cites W2250709962 @default.
- W4367016469 cites W2962801832 @default.
- W4367016469 cites W2962904552 @default.
- W4367016469 cites W2963925437 @default.
- W4367016469 cites W2964110616 @default.
- W4367016469 cites W2997945091 @default.
- W4367016469 cites W4213072767 @default.
- W4367016469 doi "https://doi.org/10.1109/nnice58320.2023.10105716" @default.
- W4367016469 hasPublicationYear "2023" @default.
- W4367016469 type Work @default.
- W4367016469 citedByCount "0" @default.
- W4367016469 crossrefType "proceedings-article" @default.
- W4367016469 hasAuthorship W4367016469A5002627939 @default.
- W4367016469 hasAuthorship W4367016469A5038264616 @default.
- W4367016469 hasConcept C121332964 @default.
- W4367016469 hasConcept C125411270 @default.
- W4367016469 hasConcept C154945302 @default.
- W4367016469 hasConcept C165801399 @default.
- W4367016469 hasConcept C203005215 @default.
- W4367016469 hasConcept C204321447 @default.
- W4367016469 hasConcept C41008148 @default.
- W4367016469 hasConcept C62520636 @default.
- W4367016469 hasConcept C66322947 @default.
- W4367016469 hasConceptScore W4367016469C121332964 @default.
- W4367016469 hasConceptScore W4367016469C125411270 @default.
- W4367016469 hasConceptScore W4367016469C154945302 @default.
- W4367016469 hasConceptScore W4367016469C165801399 @default.
- W4367016469 hasConceptScore W4367016469C203005215 @default.
- W4367016469 hasConceptScore W4367016469C204321447 @default.
- W4367016469 hasConceptScore W4367016469C41008148 @default.
- W4367016469 hasConceptScore W4367016469C62520636 @default.
- W4367016469 hasConceptScore W4367016469C66322947 @default.
- W4367016469 hasLocation W43670164691 @default.
- W4367016469 hasOpenAccess W4367016469 @default.
- W4367016469 hasPrimaryLocation W43670164691 @default.
- W4367016469 hasRelatedWork W1484029852 @default.
- W4367016469 hasRelatedWork W1512718085 @default.
- W4367016469 hasRelatedWork W1585034923 @default.
- W4367016469 hasRelatedWork W1592339875 @default.
- W4367016469 hasRelatedWork W1697423248 @default.
- W4367016469 hasRelatedWork W2422372688 @default.
- W4367016469 hasRelatedWork W2435130738 @default.
- W4367016469 hasRelatedWork W2747680751 @default.
- W4367016469 hasRelatedWork W3107474891 @default.
- W4367016469 hasRelatedWork W2610387714 @default.
- W4367016469 isParatext "false" @default.
- W4367016469 isRetracted "false" @default.
- W4367016469 workType "article" @default.