Matches in SemOpenAlex for { <https://semopenalex.org/work/W4385963463> ?p ?o ?g. }
Showing items 1 to 63 of
63
with 100 items per page.
- W4385963463 abstract "The quadratic computation complexity of self-attention has been a persistent challenge when applying Transformer models to vision tasks. Linear attention, on the other hand, offers a much more efficient alternative with its linear complexity by approximating the Softmax operation through carefully designed mapping functions. However, current linear attention approaches either suffer from significant performance degradation or introduce additional computation overhead from the mapping functions. In this paper, we propose a novel Focused Linear Attention module to achieve both high efficiency and expressiveness. Specifically, we first analyze the factors contributing to the performance degradation of linear attention from two perspectives: the focus ability and feature diversity. To overcome these limitations, we introduce a simple yet effective mapping function and an efficient rank restoration module to enhance the expressiveness of self-attention while maintaining low computation complexity. Extensive experiments show that our linear attention module is applicable to a variety of advanced vision Transformers, and achieves consistently improved performances on multiple benchmarks. Code is available at https://github.com/LeapLabTHU/FLatten-Transformer." @default.
- W4385963463 created "2023-08-18" @default.
- W4385963463 creator A5013240918 @default.
- W4385963463 creator A5024649502 @default.
- W4385963463 creator A5045569962 @default.
- W4385963463 creator A5068624597 @default.
- W4385963463 creator A5086417128 @default.
- W4385963463 date "2023-08-01" @default.
- W4385963463 modified "2023-10-16" @default.
- W4385963463 title "FLatten Transformer: Vision Transformer using Focused Linear Attention" @default.
- W4385963463 doi "https://doi.org/10.48550/arxiv.2308.00442" @default.
- W4385963463 hasPublicationYear "2023" @default.
- W4385963463 type Work @default.
- W4385963463 citedByCount "0" @default.
- W4385963463 crossrefType "posted-content" @default.
- W4385963463 hasAuthorship W4385963463A5013240918 @default.
- W4385963463 hasAuthorship W4385963463A5024649502 @default.
- W4385963463 hasAuthorship W4385963463A5045569962 @default.
- W4385963463 hasAuthorship W4385963463A5068624597 @default.
- W4385963463 hasAuthorship W4385963463A5086417128 @default.
- W4385963463 hasBestOaLocation W43859634631 @default.
- W4385963463 hasConcept C113775141 @default.
- W4385963463 hasConcept C11413529 @default.
- W4385963463 hasConcept C119599485 @default.
- W4385963463 hasConcept C127413603 @default.
- W4385963463 hasConcept C129844170 @default.
- W4385963463 hasConcept C154945302 @default.
- W4385963463 hasConcept C165801399 @default.
- W4385963463 hasConcept C2524010 @default.
- W4385963463 hasConcept C33923547 @default.
- W4385963463 hasConcept C41008148 @default.
- W4385963463 hasConcept C45374587 @default.
- W4385963463 hasConcept C66322947 @default.
- W4385963463 hasConcept C80444323 @default.
- W4385963463 hasConceptScore W4385963463C113775141 @default.
- W4385963463 hasConceptScore W4385963463C11413529 @default.
- W4385963463 hasConceptScore W4385963463C119599485 @default.
- W4385963463 hasConceptScore W4385963463C127413603 @default.
- W4385963463 hasConceptScore W4385963463C129844170 @default.
- W4385963463 hasConceptScore W4385963463C154945302 @default.
- W4385963463 hasConceptScore W4385963463C165801399 @default.
- W4385963463 hasConceptScore W4385963463C2524010 @default.
- W4385963463 hasConceptScore W4385963463C33923547 @default.
- W4385963463 hasConceptScore W4385963463C41008148 @default.
- W4385963463 hasConceptScore W4385963463C45374587 @default.
- W4385963463 hasConceptScore W4385963463C66322947 @default.
- W4385963463 hasConceptScore W4385963463C80444323 @default.
- W4385963463 hasLocation W43859634631 @default.
- W4385963463 hasOpenAccess W4385963463 @default.
- W4385963463 hasPrimaryLocation W43859634631 @default.
- W4385963463 hasRelatedWork W1572523360 @default.
- W4385963463 hasRelatedWork W2169347340 @default.
- W4385963463 hasRelatedWork W2386767533 @default.
- W4385963463 hasRelatedWork W2887599639 @default.
- W4385963463 hasRelatedWork W3157910026 @default.
- W4385963463 hasRelatedWork W4221146852 @default.
- W4385963463 hasRelatedWork W4283714288 @default.
- W4385963463 hasRelatedWork W4383501701 @default.
- W4385963463 hasRelatedWork W4385574170 @default.
- W4385963463 hasRelatedWork W2467751138 @default.
- W4385963463 isParatext "false" @default.
- W4385963463 isRetracted "false" @default.
- W4385963463 workType "article" @default.