Matches in SemOpenAlex for { <https://semopenalex.org/work/W4312349930> ?p ?o ?g. }
Showing items 1 to 95 of
95
with 100 items per page.
- W4312349930 abstract "We present techniques for scaling Swin Transformer [35] up to 3 billion parameters and making it capable of training with images of up to 1,536x1,536 resolution. By scaling up capacity and resolution, Swin Transformer sets new records on four representative vision benchmarks: 84.0% top-1 accuracy on ImageNet- V2 image classification, 63.1 / 54.4 box / mask mAP on COCO object detection, 59.9 mIoU on ADE20K semantic segmentation, and 86.8% top-1 accuracy on Kinetics-400 video action classification. We tackle issues of training instability, and study how to effectively transfer models pre-trained at low resolutions to higher resolution ones. To this aim, several novel technologies are proposed: 1) a residual post normalization technique and a scaled cosine attention approach to improve the stability of large vision models; 2) a log-spaced continuous position bias technique to effectively transfer models pre-trained at low-resolution images and windows to their higher-resolution counterparts. In addition, we share our crucial implementation details that lead to significant savings of GPU memory consumption and thus make it feasi-ble to train large vision models with regular GPUs. Using these techniques and self-supervised pre-training, we suc-cessfully train a strong 3 billion Swin Transformer model and effectively transfer it to various vision tasks involving high-resolution images or windows, achieving the state-of-the-art accuracy on a variety of benchmarks. Code is avail-able at https://github.com/microsoft/Swin-Transformer." @default.
- W4312349930 created "2023-01-04" @default.
- W4312349930 creator A5009248953 @default.
- W4312349930 creator A5012287052 @default.
- W4312349930 creator A5014662947 @default.
- W4312349930 creator A5014838804 @default.
- W4312349930 creator A5020141096 @default.
- W4312349930 creator A5028650738 @default.
- W4312349930 creator A5029901095 @default.
- W4312349930 creator A5039314490 @default.
- W4312349930 creator A5045527737 @default.
- W4312349930 creator A5084464680 @default.
- W4312349930 creator A5084466653 @default.
- W4312349930 creator A5088888083 @default.
- W4312349930 date "2022-06-01" @default.
- W4312349930 modified "2023-10-17" @default.
- W4312349930 title "Swin Transformer V2: Scaling Up Capacity and Resolution" @default.
- W4312349930 cites W2097117768 @default.
- W4312349930 cites W2108598243 @default.
- W4312349930 cites W2112796928 @default.
- W4312349930 cites W2194775991 @default.
- W4312349930 cites W2964080601 @default.
- W4312349930 cites W2983446232 @default.
- W4312349930 cites W2983943451 @default.
- W4312349930 cites W3122159272 @default.
- W4312349930 cites W3214897340 @default.
- W4312349930 cites W4312804044 @default.
- W4312349930 doi "https://doi.org/10.1109/cvpr52688.2022.01170" @default.
- W4312349930 hasPublicationYear "2022" @default.
- W4312349930 type Work @default.
- W4312349930 citedByCount "224" @default.
- W4312349930 countsByYear W43123499302022 @default.
- W4312349930 countsByYear W43123499302023 @default.
- W4312349930 crossrefType "proceedings-article" @default.
- W4312349930 hasAuthorship W4312349930A5009248953 @default.
- W4312349930 hasAuthorship W4312349930A5012287052 @default.
- W4312349930 hasAuthorship W4312349930A5014662947 @default.
- W4312349930 hasAuthorship W4312349930A5014838804 @default.
- W4312349930 hasAuthorship W4312349930A5020141096 @default.
- W4312349930 hasAuthorship W4312349930A5028650738 @default.
- W4312349930 hasAuthorship W4312349930A5029901095 @default.
- W4312349930 hasAuthorship W4312349930A5039314490 @default.
- W4312349930 hasAuthorship W4312349930A5045527737 @default.
- W4312349930 hasAuthorship W4312349930A5084464680 @default.
- W4312349930 hasAuthorship W4312349930A5084466653 @default.
- W4312349930 hasAuthorship W4312349930A5088888083 @default.
- W4312349930 hasBestOaLocation W43123499302 @default.
- W4312349930 hasConcept C119599485 @default.
- W4312349930 hasConcept C127413603 @default.
- W4312349930 hasConcept C136886441 @default.
- W4312349930 hasConcept C144024400 @default.
- W4312349930 hasConcept C153180895 @default.
- W4312349930 hasConcept C154945302 @default.
- W4312349930 hasConcept C165801399 @default.
- W4312349930 hasConcept C19165224 @default.
- W4312349930 hasConcept C2524010 @default.
- W4312349930 hasConcept C31972630 @default.
- W4312349930 hasConcept C33923547 @default.
- W4312349930 hasConcept C41008148 @default.
- W4312349930 hasConcept C66322947 @default.
- W4312349930 hasConcept C89600930 @default.
- W4312349930 hasConcept C99844830 @default.
- W4312349930 hasConceptScore W4312349930C119599485 @default.
- W4312349930 hasConceptScore W4312349930C127413603 @default.
- W4312349930 hasConceptScore W4312349930C136886441 @default.
- W4312349930 hasConceptScore W4312349930C144024400 @default.
- W4312349930 hasConceptScore W4312349930C153180895 @default.
- W4312349930 hasConceptScore W4312349930C154945302 @default.
- W4312349930 hasConceptScore W4312349930C165801399 @default.
- W4312349930 hasConceptScore W4312349930C19165224 @default.
- W4312349930 hasConceptScore W4312349930C2524010 @default.
- W4312349930 hasConceptScore W4312349930C31972630 @default.
- W4312349930 hasConceptScore W4312349930C33923547 @default.
- W4312349930 hasConceptScore W4312349930C41008148 @default.
- W4312349930 hasConceptScore W4312349930C66322947 @default.
- W4312349930 hasConceptScore W4312349930C89600930 @default.
- W4312349930 hasConceptScore W4312349930C99844830 @default.
- W4312349930 hasFunder F4320307764 @default.
- W4312349930 hasLocation W43123499301 @default.
- W4312349930 hasLocation W43123499302 @default.
- W4312349930 hasOpenAccess W4312349930 @default.
- W4312349930 hasPrimaryLocation W43123499301 @default.
- W4312349930 hasRelatedWork W1669643531 @default.
- W4312349930 hasRelatedWork W2005437358 @default.
- W4312349930 hasRelatedWork W2008656436 @default.
- W4312349930 hasRelatedWork W2039154422 @default.
- W4312349930 hasRelatedWork W2122581818 @default.
- W4312349930 hasRelatedWork W2134924024 @default.
- W4312349930 hasRelatedWork W2517104666 @default.
- W4312349930 hasRelatedWork W2533072256 @default.
- W4312349930 hasRelatedWork W2895616727 @default.
- W4312349930 hasRelatedWork W2182382398 @default.
- W4312349930 isParatext "false" @default.
- W4312349930 isRetracted "false" @default.
- W4312349930 workType "article" @default.