Matches in SemOpenAlex for { <https://semopenalex.org/work/W4287551929> ?p ?o ?g. }
Showing items 1 to 65 of
65
with 100 items per page.
- W4287551929 abstract "Although Transformer models such as Google's BERT and OpenAI's GPT-3 are successful in many natural language processing tasks, training and deploying these models are costly and inefficient.Even if pre-trained models are used, deploying these models still remained a challenge due to their large size. Apart from deployment, these models take higher time during inference restricting user-friendliness. The main bottleneck is self-attention which uses quadratic time and space with respect to the sequence length. In order to reduce the quadratic time complexity of the self-attention mechanism, Linformer by Facebook's AI research team was introduced where they showed that the self-attention mechanism can be approximated by a low-rank matrix and exploiting this finding, a new method for self-attention with linear time and space complexity was proposed by them. In the Linformer, the time complexity depends on the projection mapping dimension which acts as a hyperparameter and affects the performance of the model, tuning this hyperparameter can be time-consuming. In this paper, I proposed an alternative method for self-attention with linear complexity in time and space and is independent of the projection mapping dimension. Since this method works for long sequences this can be used for images as well as audios." @default.
- W4287551929 created "2022-07-25" @default.
- W4287551929 creator A5051654842 @default.
- W4287551929 date "2020-12-16" @default.
- W4287551929 modified "2023-09-26" @default.
- W4287551929 title "Revisiting Linformer with a modified self-attention with linear complexity" @default.
- W4287551929 doi "https://doi.org/10.48550/arxiv.2101.10277" @default.
- W4287551929 hasPublicationYear "2020" @default.
- W4287551929 type Work @default.
- W4287551929 citedByCount "0" @default.
- W4287551929 crossrefType "posted-content" @default.
- W4287551929 hasAuthorship W4287551929A5051654842 @default.
- W4287551929 hasBestOaLocation W42875519291 @default.
- W4287551929 hasConcept C11413529 @default.
- W4287551929 hasConcept C119857082 @default.
- W4287551929 hasConcept C129844170 @default.
- W4287551929 hasConcept C149635348 @default.
- W4287551929 hasConcept C154945302 @default.
- W4287551929 hasConcept C179799912 @default.
- W4287551929 hasConcept C2524010 @default.
- W4287551929 hasConcept C2776214188 @default.
- W4287551929 hasConcept C2778112365 @default.
- W4287551929 hasConcept C2780513914 @default.
- W4287551929 hasConcept C311688 @default.
- W4287551929 hasConcept C33923547 @default.
- W4287551929 hasConcept C41008148 @default.
- W4287551929 hasConcept C54355233 @default.
- W4287551929 hasConcept C57493831 @default.
- W4287551929 hasConcept C80444323 @default.
- W4287551929 hasConcept C8642999 @default.
- W4287551929 hasConcept C86803240 @default.
- W4287551929 hasConceptScore W4287551929C11413529 @default.
- W4287551929 hasConceptScore W4287551929C119857082 @default.
- W4287551929 hasConceptScore W4287551929C129844170 @default.
- W4287551929 hasConceptScore W4287551929C149635348 @default.
- W4287551929 hasConceptScore W4287551929C154945302 @default.
- W4287551929 hasConceptScore W4287551929C179799912 @default.
- W4287551929 hasConceptScore W4287551929C2524010 @default.
- W4287551929 hasConceptScore W4287551929C2776214188 @default.
- W4287551929 hasConceptScore W4287551929C2778112365 @default.
- W4287551929 hasConceptScore W4287551929C2780513914 @default.
- W4287551929 hasConceptScore W4287551929C311688 @default.
- W4287551929 hasConceptScore W4287551929C33923547 @default.
- W4287551929 hasConceptScore W4287551929C41008148 @default.
- W4287551929 hasConceptScore W4287551929C54355233 @default.
- W4287551929 hasConceptScore W4287551929C57493831 @default.
- W4287551929 hasConceptScore W4287551929C80444323 @default.
- W4287551929 hasConceptScore W4287551929C8642999 @default.
- W4287551929 hasConceptScore W4287551929C86803240 @default.
- W4287551929 hasLocation W42875519291 @default.
- W4287551929 hasOpenAccess W4287551929 @default.
- W4287551929 hasPrimaryLocation W42875519291 @default.
- W4287551929 hasRelatedWork W1483824879 @default.
- W4287551929 hasRelatedWork W1543400885 @default.
- W4287551929 hasRelatedWork W2156272239 @default.
- W4287551929 hasRelatedWork W2950696215 @default.
- W4287551929 hasRelatedWork W3126524535 @default.
- W4287551929 hasRelatedWork W4287551929 @default.
- W4287551929 hasRelatedWork W4295309597 @default.
- W4287551929 hasRelatedWork W4301599260 @default.
- W4287551929 hasRelatedWork W4323894855 @default.
- W4287551929 hasRelatedWork W2179712692 @default.
- W4287551929 isParatext "false" @default.
- W4287551929 isRetracted "false" @default.
- W4287551929 workType "article" @default.