Matches in SemOpenAlex for { <https://semopenalex.org/work/W4286906902> ?p ?o ?g. }
Showing items 1 to 78 of
78
with 100 items per page.
- W4286906902 abstract "Recently, large-scale Contrastive Language-Image Pre-training (CLIP) has attracted unprecedented attention for its impressive zero-shot recognition ability and excellent transferability to downstream tasks. However, CLIP is quite data-hungry and requires 400M image-text pairs for pre-training, thereby restricting its adoption. This work proposes a novel training paradigm, Data efficient CLIP (DeCLIP), to alleviate this limitation. We demonstrate that by carefully utilizing the widespread supervision among the image-text pairs, our De-CLIP can learn generic visual features more efficiently. Instead of using the single image-text contrastive supervision, we fully exploit data potential through the use of (1) self-supervision within each modality; (2) multi-view supervision across modalities; (3) nearest-neighbor supervision from other similar pairs. Benefiting from intrinsic supervision, our DeCLIP-ResNet50 can achieve 60.4% zero-shot top1 accuracy on ImageNet, which is 0.8% above the CLIP-ResNet50 while using 7.1 x fewer data. Our DeCLIP-ResNet50 outperforms its counterpart in 8 out of 11 visual datasets when transferred to downstream tasks. Moreover, Scaling up the model and computing also works well in our framework.Our code, dataset and models are released at: https://github.com/Sense-GVT/DeCLIP" @default.
- W4286906902 created "2022-07-25" @default.
- W4286906902 creator A5028491176 @default.
- W4286906902 creator A5034461908 @default.
- W4286906902 creator A5052226444 @default.
- W4286906902 creator A5052855665 @default.
- W4286906902 creator A5075958056 @default.
- W4286906902 creator A5081844561 @default.
- W4286906902 creator A5087818121 @default.
- W4286906902 creator A5088843448 @default.
- W4286906902 date "2021-10-11" @default.
- W4286906902 modified "2023-10-14" @default.
- W4286906902 title "Supervision Exists Everywhere: A Data Efficient Contrastive Language-Image Pre-training Paradigm" @default.
- W4286906902 hasPublicationYear "2021" @default.
- W4286906902 type Work @default.
- W4286906902 citedByCount "0" @default.
- W4286906902 crossrefType "posted-content" @default.
- W4286906902 hasAuthorship W4286906902A5028491176 @default.
- W4286906902 hasAuthorship W4286906902A5034461908 @default.
- W4286906902 hasAuthorship W4286906902A5052226444 @default.
- W4286906902 hasAuthorship W4286906902A5052855665 @default.
- W4286906902 hasAuthorship W4286906902A5075958056 @default.
- W4286906902 hasAuthorship W4286906902A5081844561 @default.
- W4286906902 hasAuthorship W4286906902A5087818121 @default.
- W4286906902 hasAuthorship W4286906902A5088843448 @default.
- W4286906902 hasBestOaLocation W42869069021 @default.
- W4286906902 hasConcept C115961682 @default.
- W4286906902 hasConcept C119857082 @default.
- W4286906902 hasConcept C140331021 @default.
- W4286906902 hasConcept C144024400 @default.
- W4286906902 hasConcept C154945302 @default.
- W4286906902 hasConcept C165696696 @default.
- W4286906902 hasConcept C177264268 @default.
- W4286906902 hasConcept C199360897 @default.
- W4286906902 hasConcept C204321447 @default.
- W4286906902 hasConcept C2776502983 @default.
- W4286906902 hasConcept C2776760102 @default.
- W4286906902 hasConcept C2779903281 @default.
- W4286906902 hasConcept C2780226545 @default.
- W4286906902 hasConcept C36289849 @default.
- W4286906902 hasConcept C38652104 @default.
- W4286906902 hasConcept C41008148 @default.
- W4286906902 hasConcept C51632099 @default.
- W4286906902 hasConcept C61272859 @default.
- W4286906902 hasConceptScore W4286906902C115961682 @default.
- W4286906902 hasConceptScore W4286906902C119857082 @default.
- W4286906902 hasConceptScore W4286906902C140331021 @default.
- W4286906902 hasConceptScore W4286906902C144024400 @default.
- W4286906902 hasConceptScore W4286906902C154945302 @default.
- W4286906902 hasConceptScore W4286906902C165696696 @default.
- W4286906902 hasConceptScore W4286906902C177264268 @default.
- W4286906902 hasConceptScore W4286906902C199360897 @default.
- W4286906902 hasConceptScore W4286906902C204321447 @default.
- W4286906902 hasConceptScore W4286906902C2776502983 @default.
- W4286906902 hasConceptScore W4286906902C2776760102 @default.
- W4286906902 hasConceptScore W4286906902C2779903281 @default.
- W4286906902 hasConceptScore W4286906902C2780226545 @default.
- W4286906902 hasConceptScore W4286906902C36289849 @default.
- W4286906902 hasConceptScore W4286906902C38652104 @default.
- W4286906902 hasConceptScore W4286906902C41008148 @default.
- W4286906902 hasConceptScore W4286906902C51632099 @default.
- W4286906902 hasConceptScore W4286906902C61272859 @default.
- W4286906902 hasLocation W42869069021 @default.
- W4286906902 hasOpenAccess W4286906902 @default.
- W4286906902 hasPrimaryLocation W42869069021 @default.
- W4286906902 hasRelatedWork W10132401 @default.
- W4286906902 hasRelatedWork W13978995 @default.
- W4286906902 hasRelatedWork W2655176 @default.
- W4286906902 hasRelatedWork W3706556 @default.
- W4286906902 hasRelatedWork W7389446 @default.
- W4286906902 hasRelatedWork W7888137 @default.
- W4286906902 hasRelatedWork W8219677 @default.
- W4286906902 hasRelatedWork W8432713 @default.
- W4286906902 hasRelatedWork W10597021 @default.
- W4286906902 hasRelatedWork W4167692 @default.
- W4286906902 isParatext "false" @default.
- W4286906902 isRetracted "false" @default.
- W4286906902 workType "article" @default.