Matches in SemOpenAlex for { <https://semopenalex.org/work/W4322759993> ?p ?o ?g. }
Showing items 1 to 85 of
85
with 100 items per page.
- W4322759993 abstract "Inspired by masked language modeling (MLM) in natural language processing, masked image modeling (MIM) has been recognized as a strong and popular self-supervised pre-training method in computer vision. However, its high random mask ratio would result in two serious problems: 1) the data are not efficiently exploited, which brings inefficient pre-training (eg, 1600 epochs for MAE $vs.$ 300 epochs for the supervised), and 2) the high uncertainty and inconsistency of the pre-trained model, ie, the prediction of the same patch may be inconsistent under different mask rounds. To tackle these problems, we propose efficient masked autoencoders with self-consistency (EMAE), to improve the pre-training efficiency and increase the consistency of MIM. In particular, we progressively divide the image into K non-overlapping parts, each of which is generated by a random mask and has the same mask ratio. Then the MIM task is conducted parallelly on all parts in an iteration and generates predictions. Besides, we design a self-consistency module to further maintain the consistency of predictions of overlapping masked patches among parts. Overall, the proposed method is able to exploit the data more efficiently and obtains reliable representations. Experiments on ImageNet show that EMAE achieves even higher results with only 300 pre-training epochs under ViT-Base than MAE (1600 epochs). EMAE also consistently obtains state-of-the-art transfer performance on various downstream tasks, like object detection, and semantic segmentation." @default.
- W4322759993 created "2023-03-03" @default.
- W4322759993 creator A5000432967 @default.
- W4322759993 creator A5004258040 @default.
- W4322759993 creator A5037249009 @default.
- W4322759993 creator A5041282060 @default.
- W4322759993 creator A5045271907 @default.
- W4322759993 creator A5058420913 @default.
- W4322759993 creator A5065043476 @default.
- W4322759993 creator A5086034088 @default.
- W4322759993 creator A5089451206 @default.
- W4322759993 date "2023-02-28" @default.
- W4322759993 modified "2023-10-16" @default.
- W4322759993 title "Efficient Masked Autoencoders with Self-Consistency" @default.
- W4322759993 doi "https://doi.org/10.48550/arxiv.2302.14431" @default.
- W4322759993 hasPublicationYear "2023" @default.
- W4322759993 type Work @default.
- W4322759993 citedByCount "0" @default.
- W4322759993 crossrefType "posted-content" @default.
- W4322759993 hasAuthorship W4322759993A5000432967 @default.
- W4322759993 hasAuthorship W4322759993A5004258040 @default.
- W4322759993 hasAuthorship W4322759993A5037249009 @default.
- W4322759993 hasAuthorship W4322759993A5041282060 @default.
- W4322759993 hasAuthorship W4322759993A5045271907 @default.
- W4322759993 hasAuthorship W4322759993A5058420913 @default.
- W4322759993 hasAuthorship W4322759993A5065043476 @default.
- W4322759993 hasAuthorship W4322759993A5086034088 @default.
- W4322759993 hasAuthorship W4322759993A5089451206 @default.
- W4322759993 hasBestOaLocation W43227599931 @default.
- W4322759993 hasConcept C111919701 @default.
- W4322759993 hasConcept C115961682 @default.
- W4322759993 hasConcept C119857082 @default.
- W4322759993 hasConcept C134306372 @default.
- W4322759993 hasConcept C153180895 @default.
- W4322759993 hasConcept C154945302 @default.
- W4322759993 hasConcept C162324750 @default.
- W4322759993 hasConcept C165696696 @default.
- W4322759993 hasConcept C187736073 @default.
- W4322759993 hasConcept C204321447 @default.
- W4322759993 hasConcept C2776436953 @default.
- W4322759993 hasConcept C2780451532 @default.
- W4322759993 hasConcept C2781238097 @default.
- W4322759993 hasConcept C33923547 @default.
- W4322759993 hasConcept C37279795 @default.
- W4322759993 hasConcept C38652104 @default.
- W4322759993 hasConcept C41008148 @default.
- W4322759993 hasConcept C42058472 @default.
- W4322759993 hasConcept C89600930 @default.
- W4322759993 hasConcept C93361087 @default.
- W4322759993 hasConceptScore W4322759993C111919701 @default.
- W4322759993 hasConceptScore W4322759993C115961682 @default.
- W4322759993 hasConceptScore W4322759993C119857082 @default.
- W4322759993 hasConceptScore W4322759993C134306372 @default.
- W4322759993 hasConceptScore W4322759993C153180895 @default.
- W4322759993 hasConceptScore W4322759993C154945302 @default.
- W4322759993 hasConceptScore W4322759993C162324750 @default.
- W4322759993 hasConceptScore W4322759993C165696696 @default.
- W4322759993 hasConceptScore W4322759993C187736073 @default.
- W4322759993 hasConceptScore W4322759993C204321447 @default.
- W4322759993 hasConceptScore W4322759993C2776436953 @default.
- W4322759993 hasConceptScore W4322759993C2780451532 @default.
- W4322759993 hasConceptScore W4322759993C2781238097 @default.
- W4322759993 hasConceptScore W4322759993C33923547 @default.
- W4322759993 hasConceptScore W4322759993C37279795 @default.
- W4322759993 hasConceptScore W4322759993C38652104 @default.
- W4322759993 hasConceptScore W4322759993C41008148 @default.
- W4322759993 hasConceptScore W4322759993C42058472 @default.
- W4322759993 hasConceptScore W4322759993C89600930 @default.
- W4322759993 hasConceptScore W4322759993C93361087 @default.
- W4322759993 hasLocation W43227599931 @default.
- W4322759993 hasOpenAccess W4322759993 @default.
- W4322759993 hasPrimaryLocation W43227599931 @default.
- W4322759993 hasRelatedWork W1850639582 @default.
- W4322759993 hasRelatedWork W2010813957 @default.
- W4322759993 hasRelatedWork W2015692847 @default.
- W4322759993 hasRelatedWork W2144606566 @default.
- W4322759993 hasRelatedWork W2350879319 @default.
- W4322759993 hasRelatedWork W2353865532 @default.
- W4322759993 hasRelatedWork W2359335444 @default.
- W4322759993 hasRelatedWork W2378103970 @default.
- W4322759993 hasRelatedWork W2392301299 @default.
- W4322759993 hasRelatedWork W2901917862 @default.
- W4322759993 isParatext "false" @default.
- W4322759993 isRetracted "false" @default.
- W4322759993 workType "article" @default.