Matches in SemOpenAlex for { <https://semopenalex.org/work/W4385300816> ?p ?o ?g. }
Showing items 1 to 82 of
82
with 100 items per page.
- W4385300816 abstract "Nowadays, the self-attention mechanism has become a resound of visual feature extraction along with convolution. The transformer network composed of self-attention has developed rapidly and made remarkable achievements in visual tasks. The self-attention shows the potential to replace convolution as the primary method of visual feature extraction in ubiquitous intelligence. Nevertheless, the development of the Visual Transformer still suffer from the following problems: a) The self-attention mechanism has a low inductive bias, which leads to large data demand and a high training cost. b) The Transformer backbone network cannot adapt well to the low visual information density and performs unsatisfactorily under low resolution and small-scale datasets. To tackle the abovementioned two problems, this paper proposes a novel algorithm based on the mature Visual Transformer architecture, which is dedicated to exploring the performance potential of the Transformer network and its kernel self-attention mechanism on small-scale datasets. Specifically, we first propose a network architecture equipped with multi-coordination strategy to solve the self-attention degradation problem inherent in the existing Transformer architecture. Secondly, we introduce consistent regularization into the Transformer to make the self-attention mechanism acquire more reliable feature representation ability in the case of insufficient visual features. In the experiments, CSwin Transformer, the mainstream visual model, is selected to verify the effectiveness of the proposed method on the prevalent small datasets, and superior results are achieved. In particular, without pre-training, our accuracy on the CIFAR-100 dataset is improved by 1.24% compared to CSwin." @default.
- W4385300816 created "2023-07-28" @default.
- W4385300816 creator A5027771775 @default.
- W4385300816 creator A5034708857 @default.
- W4385300816 creator A5039890623 @default.
- W4385300816 creator A5045807606 @default.
- W4385300816 creator A5053261798 @default.
- W4385300816 creator A5086418698 @default.
- W4385300816 date "2022-12-01" @default.
- W4385300816 modified "2023-10-18" @default.
- W4385300816 title "Redesign Visual Transformer For Small Datasets" @default.
- W4385300816 cites W2963446712 @default.
- W4385300816 cites W3033009913 @default.
- W4385300816 cites W3034619943 @default.
- W4385300816 cites W3104121806 @default.
- W4385300816 cites W3138516171 @default.
- W4385300816 cites W3158065011 @default.
- W4385300816 cites W3176743688 @default.
- W4385300816 cites W3207208339 @default.
- W4385300816 cites W4206066713 @default.
- W4385300816 cites W4214493665 @default.
- W4385300816 cites W4214636423 @default.
- W4385300816 cites W4288738569 @default.
- W4385300816 cites W4313007769 @default.
- W4385300816 doi "https://doi.org/10.1109/smartworld-uic-atc-scalcom-digitaltwin-pricomp-metaverse56740.2022.00077" @default.
- W4385300816 hasPublicationYear "2022" @default.
- W4385300816 type Work @default.
- W4385300816 citedByCount "0" @default.
- W4385300816 crossrefType "proceedings-article" @default.
- W4385300816 hasAuthorship W4385300816A5027771775 @default.
- W4385300816 hasAuthorship W4385300816A5034708857 @default.
- W4385300816 hasAuthorship W4385300816A5039890623 @default.
- W4385300816 hasAuthorship W4385300816A5045807606 @default.
- W4385300816 hasAuthorship W4385300816A5053261798 @default.
- W4385300816 hasAuthorship W4385300816A5086418698 @default.
- W4385300816 hasConcept C119599485 @default.
- W4385300816 hasConcept C119857082 @default.
- W4385300816 hasConcept C123657996 @default.
- W4385300816 hasConcept C127413603 @default.
- W4385300816 hasConcept C142362112 @default.
- W4385300816 hasConcept C144133560 @default.
- W4385300816 hasConcept C153180895 @default.
- W4385300816 hasConcept C153349607 @default.
- W4385300816 hasConcept C154945302 @default.
- W4385300816 hasConcept C162853370 @default.
- W4385300816 hasConcept C165801399 @default.
- W4385300816 hasConcept C41008148 @default.
- W4385300816 hasConcept C52622490 @default.
- W4385300816 hasConcept C66322947 @default.
- W4385300816 hasConcept C86251818 @default.
- W4385300816 hasConceptScore W4385300816C119599485 @default.
- W4385300816 hasConceptScore W4385300816C119857082 @default.
- W4385300816 hasConceptScore W4385300816C123657996 @default.
- W4385300816 hasConceptScore W4385300816C127413603 @default.
- W4385300816 hasConceptScore W4385300816C142362112 @default.
- W4385300816 hasConceptScore W4385300816C144133560 @default.
- W4385300816 hasConceptScore W4385300816C153180895 @default.
- W4385300816 hasConceptScore W4385300816C153349607 @default.
- W4385300816 hasConceptScore W4385300816C154945302 @default.
- W4385300816 hasConceptScore W4385300816C162853370 @default.
- W4385300816 hasConceptScore W4385300816C165801399 @default.
- W4385300816 hasConceptScore W4385300816C41008148 @default.
- W4385300816 hasConceptScore W4385300816C52622490 @default.
- W4385300816 hasConceptScore W4385300816C66322947 @default.
- W4385300816 hasConceptScore W4385300816C86251818 @default.
- W4385300816 hasFunder F4320321001 @default.
- W4385300816 hasLocation W43853008161 @default.
- W4385300816 hasOpenAccess W4385300816 @default.
- W4385300816 hasPrimaryLocation W43853008161 @default.
- W4385300816 hasRelatedWork W1964120219 @default.
- W4385300816 hasRelatedWork W2000165426 @default.
- W4385300816 hasRelatedWork W2114557664 @default.
- W4385300816 hasRelatedWork W2144059113 @default.
- W4385300816 hasRelatedWork W2146076056 @default.
- W4385300816 hasRelatedWork W2385132419 @default.
- W4385300816 hasRelatedWork W2772780115 @default.
- W4385300816 hasRelatedWork W2811390910 @default.
- W4385300816 hasRelatedWork W2942471066 @default.
- W4385300816 hasRelatedWork W3003836766 @default.
- W4385300816 isParatext "false" @default.
- W4385300816 isRetracted "false" @default.
- W4385300816 workType "article" @default.