Matches in SemOpenAlex for { <https://semopenalex.org/work/W4362514530> ?p ?o ?g. }
Showing items 1 to 60 of
60
with 100 items per page.
- W4362514530 abstract "Large Vision-Language Foundation Models (VLFM), such as CLIP, ALIGN and Florence, are trained on large-scale datasets of image-caption pairs and achieve superior transferability and robustness on downstream tasks, but they are difficult to use in many practical applications due to their large size, high latency and fixed architectures. Unfortunately, recent work shows training a small custom VLFM for resource-limited applications is currently very difficult using public and smaller-scale data. In this paper, we introduce a new distillation mechanism (DIME-FM) that allows us to transfer the knowledge contained in large VLFMs to smaller, customized foundation models using a relatively small amount of inexpensive, unpaired images and sentences. We transfer the knowledge from the pre-trained CLIP-ViTL/14 model to a ViT-B/32 model, with only 40M public images and 28.4M unpaired public sentences. The resulting model Distill-ViT-B/32 rivals the CLIP-ViT-B/32 model pre-trained on its private WiT dataset (400M image-text pairs): Distill-ViT-B/32 achieves similar results in terms of zero-shot and linear-probing performance on both ImageNet and the ELEVATER (20 image classification tasks) benchmarks. It also displays comparable robustness when evaluated on five datasets with natural distribution shifts from ImageNet." @default.
- W4362514530 created "2023-04-06" @default.
- W4362514530 creator A5021031074 @default.
- W4362514530 creator A5047018273 @default.
- W4362514530 creator A5057613852 @default.
- W4362514530 creator A5059735251 @default.
- W4362514530 creator A5075906727 @default.
- W4362514530 creator A5091179204 @default.
- W4362514530 date "2023-03-31" @default.
- W4362514530 modified "2023-10-16" @default.
- W4362514530 title "DIME-FM: DIstilling Multimodal and Efficient Foundation Models" @default.
- W4362514530 doi "https://doi.org/10.48550/arxiv.2303.18232" @default.
- W4362514530 hasPublicationYear "2023" @default.
- W4362514530 type Work @default.
- W4362514530 citedByCount "0" @default.
- W4362514530 crossrefType "posted-content" @default.
- W4362514530 hasAuthorship W4362514530A5021031074 @default.
- W4362514530 hasAuthorship W4362514530A5047018273 @default.
- W4362514530 hasAuthorship W4362514530A5057613852 @default.
- W4362514530 hasAuthorship W4362514530A5059735251 @default.
- W4362514530 hasAuthorship W4362514530A5075906727 @default.
- W4362514530 hasAuthorship W4362514530A5091179204 @default.
- W4362514530 hasBestOaLocation W43625145301 @default.
- W4362514530 hasConcept C104317684 @default.
- W4362514530 hasConcept C119857082 @default.
- W4362514530 hasConcept C140331021 @default.
- W4362514530 hasConcept C150899416 @default.
- W4362514530 hasConcept C154945302 @default.
- W4362514530 hasConcept C185592680 @default.
- W4362514530 hasConcept C41008148 @default.
- W4362514530 hasConcept C55493867 @default.
- W4362514530 hasConcept C61272859 @default.
- W4362514530 hasConcept C63479239 @default.
- W4362514530 hasConceptScore W4362514530C104317684 @default.
- W4362514530 hasConceptScore W4362514530C119857082 @default.
- W4362514530 hasConceptScore W4362514530C140331021 @default.
- W4362514530 hasConceptScore W4362514530C150899416 @default.
- W4362514530 hasConceptScore W4362514530C154945302 @default.
- W4362514530 hasConceptScore W4362514530C185592680 @default.
- W4362514530 hasConceptScore W4362514530C41008148 @default.
- W4362514530 hasConceptScore W4362514530C55493867 @default.
- W4362514530 hasConceptScore W4362514530C61272859 @default.
- W4362514530 hasConceptScore W4362514530C63479239 @default.
- W4362514530 hasLocation W43625145301 @default.
- W4362514530 hasLocation W43625145302 @default.
- W4362514530 hasOpenAccess W4362514530 @default.
- W4362514530 hasPrimaryLocation W43625145301 @default.
- W4362514530 hasRelatedWork W2883641150 @default.
- W4362514530 hasRelatedWork W2946016983 @default.
- W4362514530 hasRelatedWork W2960456850 @default.
- W4362514530 hasRelatedWork W3015887428 @default.
- W4362514530 hasRelatedWork W3021430260 @default.
- W4362514530 hasRelatedWork W4281645081 @default.
- W4362514530 hasRelatedWork W4308262314 @default.
- W4362514530 hasRelatedWork W4312200629 @default.
- W4362514530 hasRelatedWork W4317565044 @default.
- W4362514530 hasRelatedWork W4382286161 @default.
- W4362514530 isParatext "false" @default.
- W4362514530 isRetracted "false" @default.
- W4362514530 workType "article" @default.