Matches in SemOpenAlex for { <https://semopenalex.org/work/W4328007221> ?p ?o ?g. }
- W4328007221 endingPage "12" @default.
- W4328007221 startingPage "1" @default.
- W4328007221 abstract "Self-attention mechanisms and non-local blocks have become crucial building blocks for state-of-the-art neural architectures thanks to their unparalleled ability in capturing long-range dependencies in the input. However their cost is quadratic with the number of spatial positions hence making their use impractical in many real case applications. In this work, we analyze these methods through a polynomial lens, and we show that self-attention can be seen as a special case of a 3 rd order polynomial. Within this polynomial framework, we are able to design polynomial operators capable of accessing the same data pattern of non-local and self-attention blocks while reducing the complexity from quadratic to linear. As a result, we propose two modules (Poly-NL and Poly-SA) that can be used as drop-in replacements for more-complex non-local and self-attention layers in state-of-the-art CNNs and ViT architectures. Our modules can achieve comparable, if not better, performance across a wide range of computer vision tasks while keeping a complexity equivalent to a standard linear layer." @default.
- W4328007221 created "2023-03-22" @default.
- W4328007221 creator A5004562735 @default.
- W4328007221 creator A5012529382 @default.
- W4328007221 creator A5036945599 @default.
- W4328007221 creator A5068456312 @default.
- W4328007221 creator A5073110676 @default.
- W4328007221 creator A5080553022 @default.
- W4328007221 creator A5082442223 @default.
- W4328007221 creator A5085009855 @default.
- W4328007221 date "2023-01-01" @default.
- W4328007221 modified "2023-10-06" @default.
- W4328007221 title "Linear Complexity Self-Attention with $3^{text{rd}}$ Order Polynomials" @default.
- W4328007221 cites W1641020902 @default.
- W4328007221 cites W2024165284 @default.
- W4328007221 cites W2056370875 @default.
- W4328007221 cites W2064675550 @default.
- W4328007221 cites W2097073572 @default.
- W4328007221 cites W2104657103 @default.
- W4328007221 cites W2117539524 @default.
- W4328007221 cites W2120190345 @default.
- W4328007221 cites W2136163184 @default.
- W4328007221 cites W2170653751 @default.
- W4328007221 cites W2194775991 @default.
- W4328007221 cites W2752782242 @default.
- W4328007221 cites W2883502031 @default.
- W4328007221 cites W2884585870 @default.
- W4328007221 cites W2954930822 @default.
- W4328007221 cites W2955058313 @default.
- W4328007221 cites W2963091558 @default.
- W4328007221 cites W2963150697 @default.
- W4328007221 cites W2963319519 @default.
- W4328007221 cites W2963495494 @default.
- W4328007221 cites W2963566548 @default.
- W4328007221 cites W2963807318 @default.
- W4328007221 cites W2965669158 @default.
- W4328007221 cites W2981413347 @default.
- W4328007221 cites W2982220924 @default.
- W4328007221 cites W2983902802 @default.
- W4328007221 cites W3034217077 @default.
- W4328007221 cites W3034345703 @default.
- W4328007221 cites W3034540350 @default.
- W4328007221 cites W3085139254 @default.
- W4328007221 cites W3097065222 @default.
- W4328007221 cites W3138516171 @default.
- W4328007221 cites W3159481202 @default.
- W4328007221 cites W3167260844 @default.
- W4328007221 cites W3203784466 @default.
- W4328007221 cites W4214493665 @default.
- W4328007221 cites W4214634256 @default.
- W4328007221 cites W4312610518 @default.
- W4328007221 cites W4312820606 @default.
- W4328007221 cites W4323647442 @default.
- W4328007221 cites W78159342 @default.
- W4328007221 doi "https://doi.org/10.1109/tpami.2022.3231971" @default.
- W4328007221 hasPubMedId "https://pubmed.ncbi.nlm.nih.gov/37030770" @default.
- W4328007221 hasPublicationYear "2023" @default.
- W4328007221 type Work @default.
- W4328007221 citedByCount "0" @default.
- W4328007221 crossrefType "journal-article" @default.
- W4328007221 hasAuthorship W4328007221A5004562735 @default.
- W4328007221 hasAuthorship W4328007221A5012529382 @default.
- W4328007221 hasAuthorship W4328007221A5036945599 @default.
- W4328007221 hasAuthorship W4328007221A5068456312 @default.
- W4328007221 hasAuthorship W4328007221A5073110676 @default.
- W4328007221 hasAuthorship W4328007221A5080553022 @default.
- W4328007221 hasAuthorship W4328007221A5082442223 @default.
- W4328007221 hasAuthorship W4328007221A5085009855 @default.
- W4328007221 hasConcept C11413529 @default.
- W4328007221 hasConcept C129844170 @default.
- W4328007221 hasConcept C134306372 @default.
- W4328007221 hasConcept C159985019 @default.
- W4328007221 hasConcept C192562407 @default.
- W4328007221 hasConcept C204323151 @default.
- W4328007221 hasConcept C2524010 @default.
- W4328007221 hasConcept C311688 @default.
- W4328007221 hasConcept C33923547 @default.
- W4328007221 hasConcept C41008148 @default.
- W4328007221 hasConcept C80444323 @default.
- W4328007221 hasConcept C90119067 @default.
- W4328007221 hasConceptScore W4328007221C11413529 @default.
- W4328007221 hasConceptScore W4328007221C129844170 @default.
- W4328007221 hasConceptScore W4328007221C134306372 @default.
- W4328007221 hasConceptScore W4328007221C159985019 @default.
- W4328007221 hasConceptScore W4328007221C192562407 @default.
- W4328007221 hasConceptScore W4328007221C204323151 @default.
- W4328007221 hasConceptScore W4328007221C2524010 @default.
- W4328007221 hasConceptScore W4328007221C311688 @default.
- W4328007221 hasConceptScore W4328007221C33923547 @default.
- W4328007221 hasConceptScore W4328007221C41008148 @default.
- W4328007221 hasConceptScore W4328007221C80444323 @default.
- W4328007221 hasConceptScore W4328007221C90119067 @default.
- W4328007221 hasLocation W43280072211 @default.
- W4328007221 hasLocation W43280072212 @default.
- W4328007221 hasOpenAccess W4328007221 @default.
- W4328007221 hasPrimaryLocation W43280072211 @default.
- W4328007221 hasRelatedWork W2041119006 @default.
- W4328007221 hasRelatedWork W2060112755 @default.