Matches in SemOpenAlex for { <https://semopenalex.org/work/W3146366485> ?p ?o ?g. }
- W3146366485 endingPage "62" @default.
- W3146366485 startingPage "48" @default.
- W3146366485 abstract "Attention has arguably become one of the most important concepts in the deep learning field. It is inspired by the biological systems of humans that tend to focus on the distinctive parts when processing large amounts of information. With the development of deep neural networks, attention mechanism has been widely used in diverse application domains. This paper aims to give an overview of the state-of-the-art attention models proposed in recent years. Toward a better general understanding of attention mechanisms, we define a unified model that is suitable for most attention structures. Each step of the attention mechanism implemented in the model is described in detail. Furthermore, we classify existing attention models according to four criteria: the softness of attention, forms of input feature, input representation, and output representation. Besides, we summarize network architectures used in conjunction with the attention mechanism and describe some typical applications of attention mechanism. Finally, we discuss the interpretability that attention brings to deep learning and present its potential future trends." @default.
- W3146366485 created "2021-04-13" @default.
- W3146366485 creator A5006580423 @default.
- W3146366485 creator A5046728432 @default.
- W3146366485 creator A5088298875 @default.
- W3146366485 date "2021-09-01" @default.
- W3146366485 modified "2023-10-14" @default.
- W3146366485 title "A review on the attention mechanism of deep learning" @default.
- W3146366485 cites W1902237438 @default.
- W3146366485 cites W2010041218 @default.
- W3146366485 cites W2019370496 @default.
- W3146366485 cites W2064675550 @default.
- W3146366485 cites W2089597841 @default.
- W3146366485 cites W2119717200 @default.
- W3146366485 cites W2122710056 @default.
- W3146366485 cites W2128272608 @default.
- W3146366485 cites W2131774270 @default.
- W3146366485 cites W2139501017 @default.
- W3146366485 cites W2157331557 @default.
- W3146366485 cites W2250539671 @default.
- W3146366485 cites W2327501763 @default.
- W3146366485 cites W2470673105 @default.
- W3146366485 cites W2511929605 @default.
- W3146366485 cites W2575842049 @default.
- W3146366485 cites W2594990650 @default.
- W3146366485 cites W2604205681 @default.
- W3146366485 cites W2750499125 @default.
- W3146366485 cites W2760103715 @default.
- W3146366485 cites W2787802257 @default.
- W3146366485 cites W2788647998 @default.
- W3146366485 cites W2788701627 @default.
- W3146366485 cites W2788997749 @default.
- W3146366485 cites W2789028911 @default.
- W3146366485 cites W2805121932 @default.
- W3146366485 cites W2808281579 @default.
- W3146366485 cites W2808310571 @default.
- W3146366485 cites W2868755588 @default.
- W3146366485 cites W2884585870 @default.
- W3146366485 cites W2885822952 @default.
- W3146366485 cites W2891430112 @default.
- W3146366485 cites W2891635683 @default.
- W3146366485 cites W2891778157 @default.
- W3146366485 cites W2892381115 @default.
- W3146366485 cites W2895340641 @default.
- W3146366485 cites W2898658722 @default.
- W3146366485 cites W2900120976 @default.
- W3146366485 cites W2910058009 @default.
- W3146366485 cites W2912614123 @default.
- W3146366485 cites W2913939882 @default.
- W3146366485 cites W2914713622 @default.
- W3146366485 cites W2914767245 @default.
- W3146366485 cites W2922509574 @default.
- W3146366485 cites W2946794439 @default.
- W3146366485 cites W2950568498 @default.
- W3146366485 cites W2950768109 @default.
- W3146366485 cites W2955058313 @default.
- W3146366485 cites W2955576045 @default.
- W3146366485 cites W2962739339 @default.
- W3146366485 cites W2962772482 @default.
- W3146366485 cites W2962822108 @default.
- W3146366485 cites W2963091558 @default.
- W3146366485 cites W2963168371 @default.
- W3146366485 cites W2963358003 @default.
- W3146366485 cites W2963420686 @default.
- W3146366485 cites W2963436889 @default.
- W3146366485 cites W2963448850 @default.
- W3146366485 cites W2963495494 @default.
- W3146366485 cites W2963598809 @default.
- W3146366485 cites W2964089206 @default.
- W3146366485 cites W2964110616 @default.
- W3146366485 cites W2964189376 @default.
- W3146366485 cites W2965901996 @default.
- W3146366485 cites W2970726176 @default.
- W3146366485 cites W2971370265 @default.
- W3146366485 cites W2975802621 @default.
- W3146366485 cites W2981107167 @default.
- W3146366485 cites W2981689412 @default.
- W3146366485 cites W2982220924 @default.
- W3146366485 cites W2983902802 @default.
- W3146366485 cites W2987761193 @default.
- W3146366485 cites W3004639522 @default.
- W3146366485 cites W3034444624 @default.
- W3146366485 cites W3045692700 @default.
- W3146366485 cites W3046154154 @default.
- W3146366485 cites W3049686946 @default.
- W3146366485 cites W3082334751 @default.
- W3146366485 cites W3083239214 @default.
- W3146366485 cites W3091765343 @default.
- W3146366485 cites W3099311608 @default.
- W3146366485 cites W3104486441 @default.
- W3146366485 cites W3105886036 @default.
- W3146366485 cites W3111817041 @default.
- W3146366485 cites W3115668312 @default.
- W3146366485 cites W3118635606 @default.
- W3146366485 cites W3126983570 @default.
- W3146366485 cites W3133854053 @default.
- W3146366485 cites W3134666949 @default.
- W3146366485 cites W3148388528 @default.