FRINK Linked Data Fragments server

Matches in SemOpenAlex for { <https://semopenalex.org/work/W3130878351> ?p ?o ?g. }

• W3130878351 abstract "Self-attention, as the key block of transformers, is a powerful mechanism for extracting features from the inputs. In essence, what self-attention does is to infer the pairwise relations between the elements of the inputs, and modify the inputs by propagating information between input pairs. As a result, it maps inputs to N outputs and casts a quadratic $O(N^2)$ memory and time complexity. We propose centroid attention, a generalization of self-attention that maps N inputs to M outputs $(Mleq N)$, such that the key information in the inputs are summarized in the smaller number of outputs (called centroids). We design centroid attention by amortizing the gradient descent update rule of a clustering objective function on the inputs, which reveals an underlying connection between attention and clustering. By compressing the inputs to the centroids, we extract the key information useful for prediction and also reduce the computation of the attention module and the subsequent layers. We apply our method to various applications, including abstractive text summarization, 3D vision, and image processing. Empirical results demonstrate the effectiveness of our method over the standard transformers." @default.
• W3130878351 created "2021-03-01" @default.
• W3130878351 creator A5024864770 @default.
• W3130878351 creator A5057329072 @default.
• W3130878351 creator A5087616209 @default.
• W3130878351 date "2021-02-17" @default.
• W3130878351 modified "2023-09-23" @default.
• W3130878351 title "Centroid Transformers: Learning to Abstract with Attention." @default.
• W3130878351 cites W1686810756 @default.
• W3130878351 cites W1843891098 @default.
• W3130878351 cites W1920022804 @default.
• W3130878351 cites W2108598243 @default.
• W3130878351 cites W2123045220 @default.
• W3130878351 cites W2154652894 @default.
• W3130878351 cites W2190691619 @default.
• W3130878351 cites W2533545350 @default.
• W3130878351 cites W2560609797 @default.
• W3130878351 cites W2601564443 @default.
• W3130878351 cites W2624503621 @default.
• W3130878351 cites W2751777443 @default.
• W3130878351 cites W2784962210 @default.
• W3130878351 cites W2785994986 @default.
• W3130878351 cites W2903024257 @default.
• W3130878351 cites W2930556772 @default.
• W3130878351 cites W2943845043 @default.
• W3130878351 cites W2948798935 @default.
• W3130878351 cites W2953273646 @default.
• W3130878351 cites W2962712961 @default.
• W3130878351 cites W2962851485 @default.
• W3130878351 cites W2962852342 @default.
• W3130878351 cites W2963403868 @default.
• W3130878351 cites W2963840672 @default.
• W3130878351 cites W2963877826 @default.
• W3130878351 cites W2963929190 @default.
• W3130878351 cites W2964228567 @default.
• W3130878351 cites W2966661 @default.
• W3130878351 cites W2979750740 @default.
• W3130878351 cites W2994673210 @default.
• W3130878351 cites W2997517014 @default.
• W3130878351 cites W3012494314 @default.
• W3130878351 cites W3015468748 @default.
• W3130878351 cites W3033529678 @default.
• W3130878351 cites W3034573343 @default.
• W3130878351 cites W3034664537 @default.
• W3130878351 cites W3034742519 @default.
• W3130878351 cites W3047517563 @default.
• W3130878351 cites W3094502228 @default.
• W3130878351 cites W3096609285 @default.
• W3130878351 cites W3105966348 @default.
• W3130878351 cites W3107118900 @default.
• W3130878351 cites W3116489684 @default.
• W3130878351 cites W3120633509 @default.
• W3130878351 hasPublicationYear "2021" @default.
• W3130878351 type Work @default.
• W3130878351 sameAs 3130878351 @default.
• W3130878351 citedByCount "2" @default.
• W3130878351 countsByYear W31308783512021 @default.
• W3130878351 crossrefType "posted-content" @default.
• W3130878351 hasAuthorship W3130878351A5024864770 @default.
• W3130878351 hasAuthorship W3130878351A5057329072 @default.
• W3130878351 hasAuthorship W3130878351A5087616209 @default.
• W3130878351 hasConcept C11413529 @default.
• W3130878351 hasConcept C121332964 @default.
• W3130878351 hasConcept C124101348 @default.
• W3130878351 hasConcept C129844170 @default.
• W3130878351 hasConcept C146599234 @default.
• W3130878351 hasConcept C153180895 @default.
• W3130878351 hasConcept C154945302 @default.
• W3130878351 hasConcept C165801399 @default.
• W3130878351 hasConcept C170858558 @default.
• W3130878351 hasConcept C184898388 @default.
• W3130878351 hasConcept C2524010 @default.
• W3130878351 hasConcept C26517878 @default.
• W3130878351 hasConcept C33923547 @default.
• W3130878351 hasConcept C38652104 @default.
• W3130878351 hasConcept C41008148 @default.
• W3130878351 hasConcept C45374587 @default.
• W3130878351 hasConcept C62520636 @default.
• W3130878351 hasConcept C66322947 @default.
• W3130878351 hasConcept C73555534 @default.
• W3130878351 hasConceptScore W3130878351C11413529 @default.
• W3130878351 hasConceptScore W3130878351C121332964 @default.
• W3130878351 hasConceptScore W3130878351C124101348 @default.
• W3130878351 hasConceptScore W3130878351C129844170 @default.
• W3130878351 hasConceptScore W3130878351C146599234 @default.
• W3130878351 hasConceptScore W3130878351C153180895 @default.
• W3130878351 hasConceptScore W3130878351C154945302 @default.
• W3130878351 hasConceptScore W3130878351C165801399 @default.
• W3130878351 hasConceptScore W3130878351C170858558 @default.
• W3130878351 hasConceptScore W3130878351C184898388 @default.
• W3130878351 hasConceptScore W3130878351C2524010 @default.
• W3130878351 hasConceptScore W3130878351C26517878 @default.
• W3130878351 hasConceptScore W3130878351C33923547 @default.
• W3130878351 hasConceptScore W3130878351C38652104 @default.
• W3130878351 hasConceptScore W3130878351C41008148 @default.
• W3130878351 hasConceptScore W3130878351C45374587 @default.
• W3130878351 hasConceptScore W3130878351C62520636 @default.
• W3130878351 hasConceptScore W3130878351C66322947 @default.
• W3130878351 hasConceptScore W3130878351C73555534 @default.
• W3130878351 hasLocation W31308783511 @default.

• next