Matches in SemOpenAlex for { <https://semopenalex.org/work/W4287164561> ?p ?o ?g. }
Showing items 1 to 67 of
67
with 100 items per page.
- W4287164561 abstract "We introduce Attention Free Transformer (AFT), an efficient variant of Transformers that eliminates the need for dot product self attention. In an AFT layer, the key and value are first combined with a set of learned position biases, the result of which is multiplied with the query in an element-wise fashion. This new operation has a memory complexity linear w.r.t. both the context size and the dimension of features, making it compatible to both large input and model sizes. We also introduce AFT-local and AFT-conv, two model variants that take advantage of the idea of locality and spatial weight sharing while maintaining global connectivity. We conduct extensive experiments on two autoregressive modeling tasks (CIFAR10 and Enwik8) as well as an image recognition task (ImageNet-1K classification). We show that AFT demonstrates competitive performance on all the benchmarks, while providing excellent efficiency at the same time." @default.
- W4287164561 created "2022-07-25" @default.
- W4287164561 creator A5003628305 @default.
- W4287164561 creator A5029638208 @default.
- W4287164561 creator A5031329242 @default.
- W4287164561 creator A5033616704 @default.
- W4287164561 creator A5043808400 @default.
- W4287164561 creator A5057849732 @default.
- W4287164561 creator A5085080733 @default.
- W4287164561 date "2021-05-28" @default.
- W4287164561 modified "2023-09-28" @default.
- W4287164561 title "An Attention Free Transformer" @default.
- W4287164561 doi "https://doi.org/10.48550/arxiv.2105.14103" @default.
- W4287164561 hasPublicationYear "2021" @default.
- W4287164561 type Work @default.
- W4287164561 citedByCount "0" @default.
- W4287164561 crossrefType "posted-content" @default.
- W4287164561 hasAuthorship W4287164561A5003628305 @default.
- W4287164561 hasAuthorship W4287164561A5029638208 @default.
- W4287164561 hasAuthorship W4287164561A5031329242 @default.
- W4287164561 hasAuthorship W4287164561A5033616704 @default.
- W4287164561 hasAuthorship W4287164561A5043808400 @default.
- W4287164561 hasAuthorship W4287164561A5057849732 @default.
- W4287164561 hasAuthorship W4287164561A5085080733 @default.
- W4287164561 hasBestOaLocation W42871645611 @default.
- W4287164561 hasConcept C119599485 @default.
- W4287164561 hasConcept C119857082 @default.
- W4287164561 hasConcept C127413603 @default.
- W4287164561 hasConcept C138885662 @default.
- W4287164561 hasConcept C149782125 @default.
- W4287164561 hasConcept C154945302 @default.
- W4287164561 hasConcept C159877910 @default.
- W4287164561 hasConcept C165801399 @default.
- W4287164561 hasConcept C2779808786 @default.
- W4287164561 hasConcept C33923547 @default.
- W4287164561 hasConcept C41008148 @default.
- W4287164561 hasConcept C41895202 @default.
- W4287164561 hasConcept C66322947 @default.
- W4287164561 hasConceptScore W4287164561C119599485 @default.
- W4287164561 hasConceptScore W4287164561C119857082 @default.
- W4287164561 hasConceptScore W4287164561C127413603 @default.
- W4287164561 hasConceptScore W4287164561C138885662 @default.
- W4287164561 hasConceptScore W4287164561C149782125 @default.
- W4287164561 hasConceptScore W4287164561C154945302 @default.
- W4287164561 hasConceptScore W4287164561C159877910 @default.
- W4287164561 hasConceptScore W4287164561C165801399 @default.
- W4287164561 hasConceptScore W4287164561C2779808786 @default.
- W4287164561 hasConceptScore W4287164561C33923547 @default.
- W4287164561 hasConceptScore W4287164561C41008148 @default.
- W4287164561 hasConceptScore W4287164561C41895202 @default.
- W4287164561 hasConceptScore W4287164561C66322947 @default.
- W4287164561 hasLocation W42871645611 @default.
- W4287164561 hasOpenAccess W4287164561 @default.
- W4287164561 hasPrimaryLocation W42871645611 @default.
- W4287164561 hasRelatedWork W1548317368 @default.
- W4287164561 hasRelatedWork W2363648756 @default.
- W4287164561 hasRelatedWork W2381880241 @default.
- W4287164561 hasRelatedWork W2576147416 @default.
- W4287164561 hasRelatedWork W2589307556 @default.
- W4287164561 hasRelatedWork W2626799276 @default.
- W4287164561 hasRelatedWork W2886384632 @default.
- W4287164561 hasRelatedWork W2961085424 @default.
- W4287164561 hasRelatedWork W2970121907 @default.
- W4287164561 hasRelatedWork W4366525293 @default.
- W4287164561 isParatext "false" @default.
- W4287164561 isRetracted "false" @default.
- W4287164561 workType "article" @default.