Matches in SemOpenAlex for { <https://semopenalex.org/work/W2963406157> ?p ?o ?g. }
Showing items 1 to 89 of
89
with 100 items per page.
- W2963406157 abstract "Multi-head attention is appealing for its ability to jointly extract different types of information from multiple representation subspaces. Concerning the information aggregation, a common practice is to use a concatenation followed by a linear transformation, which may not fully exploit the expressiveness of multi-head attention. In this work, we propose to improve the information aggregation for multi-head attention with a more powerful routing-by-agreement algorithm. Specifically, the routing algorithm iteratively updates the proportion of how much a part (i.e. the distinct information learned from a specific subspace) should be assigned to a whole (i.e. the final output representation), based on the agreement between parts and wholes. Experimental results on linguistic probing tasks and machine translation tasks prove the superiority of the advanced information aggregation over the standard linear transformation." @default.
- W2963406157 created "2019-07-30" @default.
- W2963406157 creator A5016804474 @default.
- W2963406157 creator A5028040391 @default.
- W2963406157 creator A5038315768 @default.
- W2963406157 creator A5039977782 @default.
- W2963406157 creator A5044084857 @default.
- W2963406157 creator A5052883326 @default.
- W2963406157 date "2019-01-01" @default.
- W2963406157 modified "2023-09-30" @default.
- W2963406157 title "Information Aggregation for Multi-Head Attention with Routing-by-Agreement" @default.
- W2963406157 cites W1514535095 @default.
- W2963406157 cites W1902237438 @default.
- W2963406157 cites W2101105183 @default.
- W2963406157 cites W222053410 @default.
- W2963406157 cites W2563574619 @default.
- W2963406157 cites W2767989436 @default.
- W2963406157 cites W2775143585 @default.
- W2963406157 cites W2785994986 @default.
- W2963406157 cites W2797472209 @default.
- W2963406157 cites W2798761464 @default.
- W2963406157 cites W2799124508 @default.
- W2963406157 cites W2888539709 @default.
- W2963406157 cites W2899423466 @default.
- W2963406157 cites W2912351236 @default.
- W2963406157 cites W2962739339 @default.
- W2963406157 cites W2962784628 @default.
- W2963406157 cites W2962788148 @default.
- W2963406157 cites W2962822108 @default.
- W2963406157 cites W2962853356 @default.
- W2963406157 cites W2962931466 @default.
- W2963406157 cites W2963341956 @default.
- W2963406157 cites W2963383024 @default.
- W2963406157 cites W2963386218 @default.
- W2963406157 cites W2963403868 @default.
- W2963406157 cites W2963499089 @default.
- W2963406157 cites W2963703618 @default.
- W2963406157 cites W2963717374 @default.
- W2963406157 cites W2964265128 @default.
- W2963406157 cites W2964308564 @default.
- W2963406157 cites W2966661 @default.
- W2963406157 cites W854541894 @default.
- W2963406157 doi "https://doi.org/10.18653/v1/n19-1359" @default.
- W2963406157 hasPublicationYear "2019" @default.
- W2963406157 type Work @default.
- W2963406157 sameAs 2963406157 @default.
- W2963406157 citedByCount "30" @default.
- W2963406157 countsByYear W29634061572019 @default.
- W2963406157 countsByYear W29634061572020 @default.
- W2963406157 countsByYear W29634061572021 @default.
- W2963406157 countsByYear W29634061572022 @default.
- W2963406157 crossrefType "proceedings-article" @default.
- W2963406157 hasAuthorship W2963406157A5016804474 @default.
- W2963406157 hasAuthorship W2963406157A5028040391 @default.
- W2963406157 hasAuthorship W2963406157A5038315768 @default.
- W2963406157 hasAuthorship W2963406157A5039977782 @default.
- W2963406157 hasAuthorship W2963406157A5044084857 @default.
- W2963406157 hasAuthorship W2963406157A5052883326 @default.
- W2963406157 hasBestOaLocation W29634061572 @default.
- W2963406157 hasConcept C114793014 @default.
- W2963406157 hasConcept C127313418 @default.
- W2963406157 hasConcept C2780312720 @default.
- W2963406157 hasConcept C31258907 @default.
- W2963406157 hasConcept C41008148 @default.
- W2963406157 hasConcept C74172769 @default.
- W2963406157 hasConceptScore W2963406157C114793014 @default.
- W2963406157 hasConceptScore W2963406157C127313418 @default.
- W2963406157 hasConceptScore W2963406157C2780312720 @default.
- W2963406157 hasConceptScore W2963406157C31258907 @default.
- W2963406157 hasConceptScore W2963406157C41008148 @default.
- W2963406157 hasConceptScore W2963406157C74172769 @default.
- W2963406157 hasLocation W29634061571 @default.
- W2963406157 hasLocation W29634061572 @default.
- W2963406157 hasOpenAccess W2963406157 @default.
- W2963406157 hasPrimaryLocation W29634061571 @default.
- W2963406157 hasRelatedWork W1484196235 @default.
- W2963406157 hasRelatedWork W1583271986 @default.
- W2963406157 hasRelatedWork W2356045539 @default.
- W2963406157 hasRelatedWork W2359300535 @default.
- W2963406157 hasRelatedWork W2365493740 @default.
- W2963406157 hasRelatedWork W2369318675 @default.
- W2963406157 hasRelatedWork W2387766014 @default.
- W2963406157 hasRelatedWork W2925451740 @default.
- W2963406157 hasRelatedWork W2994416637 @default.
- W2963406157 hasRelatedWork W3106476875 @default.
- W2963406157 isParatext "false" @default.
- W2963406157 isRetracted "false" @default.
- W2963406157 magId "2963406157" @default.
- W2963406157 workType "article" @default.