FRINK Linked Data Fragments server

Matches in SemOpenAlex for { <https://semopenalex.org/work/W3192125374> ?p ?o ?g. }

• W3192125374 abstract "Transformer has been widely used for self-supervised pre-training in Natural Language Processing (NLP) and achieved great success. However, it has not been fully explored in visual self-supervised learning. Meanwhile, previous methods only consider the high-level feature and learning representation from a global perspective, which may fail to transfer to the downstream dense prediction tasks focusing on local features. In this paper, we present a novel Masked Self-supervised Transformer approach named MST, which can explicitly capture the local context of an image while preserving the global semantic information. Specifically, inspired by the Masked Language Modeling (MLM) in NLP, we propose a masked token strategy based on the multi-head self-attention map, which dynamically masks some tokens of local patches without damaging the crucial structure for self-supervised learning. More importantly, the masked tokens together with the remaining tokens are further recovered by a global image decoder, which preserves the spatial information of the image and is more friendly to the downstream dense prediction tasks. The experiments on multiple datasets demonstrate the effectiveness and generality of the proposed method. For instance, MST achieves Top-1 accuracy of 76.9% with DeiT-S only using 300-epoch pre-training by linear evaluation, which outperforms supervised methods with the same epoch by 0.4% and its comparable variant DINO by 1.0%. For dense prediction tasks, MST also achieves 42.7% mAP on MS COCO object detection and 74.04% mIoU on Cityscapes segmentation only with 100-epoch pre-training." @default.
• W3192125374 created "2021-08-16" @default.
• W3192125374 creator A5000205902 @default.
• W3192125374 creator A5000432967 @default.
• W3192125374 creator A5004258040 @default.
• W3192125374 creator A5012022983 @default.
• W3192125374 creator A5024522346 @default.
• W3192125374 creator A5037731437 @default.
• W3192125374 creator A5045271907 @default.
• W3192125374 creator A5058420913 @default.
• W3192125374 creator A5063207430 @default.
• W3192125374 creator A5065043476 @default.
• W3192125374 creator A5086034088 @default.
• W3192125374 date "2021-06-10" @default.
• W3192125374 modified "2023-09-26" @default.
• W3192125374 title "MST: Masked Self-Supervised Transformer for Visual Representation" @default.
• W3192125374 cites W1861492603 @default.
• W3192125374 cites W2108598243 @default.
• W3192125374 cites W2194775991 @default.
• W3192125374 cites W2340897893 @default.
• W3192125374 cites W2518108298 @default.
• W3192125374 cites W2622263826 @default.
• W3192125374 cites W2798991696 @default.
• W3192125374 cites W2842511635 @default.
• W3192125374 cites W2887997457 @default.
• W3192125374 cites W2908510526 @default.
• W3192125374 cites W2963150697 @default.
• W3192125374 cites W2963341956 @default.
• W3192125374 cites W2963403868 @default.
• W3192125374 cites W2987283559 @default.
• W3192125374 cites W3005680577 @default.
• W3192125374 cites W3009561768 @default.
• W3192125374 cites W3034445277 @default.
• W3192125374 cites W3095121901 @default.
• W3192125374 cites W3100859887 @default.
• W3192125374 cites W3101821705 @default.
• W3192125374 cites W3106528393 @default.
• W3192125374 cites W3106539090 @default.
• W3192125374 cites W3110674625 @default.
• W3192125374 cites W3116489684 @default.
• W3192125374 cites W3119786062 @default.
• W3192125374 cites W3122240496 @default.
• W3192125374 cites W3123072794 @default.
• W3192125374 cites W3130807600 @default.
• W3192125374 cites W3138516171 @default.
• W3192125374 cites W3145450063 @default.
• W3192125374 cites W3160566314 @default.
• W3192125374 cites W3170841864 @default.
• W3192125374 cites W3172615411 @default.
• W3192125374 cites W3204138855 @default.
• W3192125374 cites W3108262825 @default.
• W3192125374 hasPublicationYear "2021" @default.
• W3192125374 type Work @default.
• W3192125374 sameAs 3192125374 @default.
• W3192125374 citedByCount "2" @default.
• W3192125374 countsByYear W31921253742020 @default.
• W3192125374 countsByYear W31921253742021 @default.
• W3192125374 crossrefType "posted-content" @default.
• W3192125374 hasAuthorship W3192125374A5000205902 @default.
• W3192125374 hasAuthorship W3192125374A5000432967 @default.
• W3192125374 hasAuthorship W3192125374A5004258040 @default.
• W3192125374 hasAuthorship W3192125374A5012022983 @default.
• W3192125374 hasAuthorship W3192125374A5024522346 @default.
• W3192125374 hasAuthorship W3192125374A5037731437 @default.
• W3192125374 hasAuthorship W3192125374A5045271907 @default.
• W3192125374 hasAuthorship W3192125374A5058420913 @default.
• W3192125374 hasAuthorship W3192125374A5063207430 @default.
• W3192125374 hasAuthorship W3192125374A5065043476 @default.
• W3192125374 hasAuthorship W3192125374A5086034088 @default.
• W3192125374 hasConcept C119857082 @default.
• W3192125374 hasConcept C121332964 @default.
• W3192125374 hasConcept C136389625 @default.
• W3192125374 hasConcept C153180895 @default.
• W3192125374 hasConcept C154945302 @default.
• W3192125374 hasConcept C15744967 @default.
• W3192125374 hasConcept C165801399 @default.
• W3192125374 hasConcept C2780767217 @default.
• W3192125374 hasConcept C38652104 @default.
• W3192125374 hasConcept C41008148 @default.
• W3192125374 hasConcept C48145219 @default.
• W3192125374 hasConcept C50644808 @default.
• W3192125374 hasConcept C542102704 @default.
• W3192125374 hasConcept C59404180 @default.
• W3192125374 hasConcept C62520636 @default.
• W3192125374 hasConcept C66322947 @default.
• W3192125374 hasConcept C89600930 @default.
• W3192125374 hasConceptScore W3192125374C119857082 @default.
• W3192125374 hasConceptScore W3192125374C121332964 @default.
• W3192125374 hasConceptScore W3192125374C136389625 @default.
• W3192125374 hasConceptScore W3192125374C153180895 @default.
• W3192125374 hasConceptScore W3192125374C154945302 @default.
• W3192125374 hasConceptScore W3192125374C15744967 @default.
• W3192125374 hasConceptScore W3192125374C165801399 @default.
• W3192125374 hasConceptScore W3192125374C2780767217 @default.
• W3192125374 hasConceptScore W3192125374C38652104 @default.
• W3192125374 hasConceptScore W3192125374C41008148 @default.
• W3192125374 hasConceptScore W3192125374C48145219 @default.
• W3192125374 hasConceptScore W3192125374C50644808 @default.
• W3192125374 hasConceptScore W3192125374C542102704 @default.
• W3192125374 hasConceptScore W3192125374C59404180 @default.

• next