Matches in SemOpenAlex for { <https://semopenalex.org/work/W4386065512> ?p ?o ?g. }
Showing items 1 to 99 of
99
with 100 items per page.
- W4386065512 abstract "Scaling up neural networks has led to remarkable performance across a wide range of tasks. Moreover, performance often follows reliable scaling laws as a function of training set size, model size, and compute, which offers valuable guidance as large-scale experiments are becoming increasingly expensive. However, previous work on scaling laws has primarily used private data & models or focused on uni-modal language or vision learning. To address these limitations, we investigate scaling laws for contrastive language-image pre-training (CLIP) with the public LAION dataset and the open-source OpenCLIP repository. Our large-scale experiments involve models trained on up to two billion image-text pairs and identify power law scaling for multiple downstream tasks including zero-shot classification, retrieval, linear probing, and end-to-end fine-tuning. We find that the training distribution plays a key role in scaling laws as the OpenAI and OpenCLIP models exhibit different scaling behavior despite identical model architectures and similar training recipes. We open-source our evaluation workflow and all models, including the largest public CLIP models, to ensure reproducibility and make scaling laws research more accessible. Source code and instructions to reproduce this study is available at https://github.eom/LAION-AI/sealing-laws-openelip." @default.
- W4386065512 created "2023-08-23" @default.
- W4386065512 creator A5001590363 @default.
- W4386065512 creator A5019618900 @default.
- W4386065512 creator A5048883432 @default.
- W4386065512 creator A5050394079 @default.
- W4386065512 creator A5056602151 @default.
- W4386065512 creator A5068360032 @default.
- W4386065512 creator A5079129211 @default.
- W4386065512 creator A5085870953 @default.
- W4386065512 creator A5091825919 @default.
- W4386065512 date "2023-06-01" @default.
- W4386065512 modified "2023-09-27" @default.
- W4386065512 title "Reproducible Scaling Laws for Contrastive Language-Image Learning" @default.
- W4386065512 cites W1977766639 @default.
- W4386065512 cites W2108598243 @default.
- W4386065512 cites W2117876524 @default.
- W4386065512 cites W2138011018 @default.
- W4386065512 cites W2185175083 @default.
- W4386065512 cites W2250384498 @default.
- W4386065512 cites W2277195237 @default.
- W4386065512 cites W2804935296 @default.
- W4386065512 cites W2886641317 @default.
- W4386065512 cites W2962843773 @default.
- W4386065512 cites W2963026768 @default.
- W4386065512 cites W2963518130 @default.
- W4386065512 cites W2964194231 @default.
- W4386065512 cites W3037492894 @default.
- W4386065512 cites W3086105743 @default.
- W4386065512 cites W3168545914 @default.
- W4386065512 cites W3172942063 @default.
- W4386065512 cites W3198675127 @default.
- W4386065512 cites W3213454282 @default.
- W4386065512 cites W4225581307 @default.
- W4386065512 cites W4312933868 @default.
- W4386065512 doi "https://doi.org/10.1109/cvpr52729.2023.00276" @default.
- W4386065512 hasPublicationYear "2023" @default.
- W4386065512 type Work @default.
- W4386065512 citedByCount "0" @default.
- W4386065512 crossrefType "proceedings-article" @default.
- W4386065512 hasAuthorship W4386065512A5001590363 @default.
- W4386065512 hasAuthorship W4386065512A5019618900 @default.
- W4386065512 hasAuthorship W4386065512A5048883432 @default.
- W4386065512 hasAuthorship W4386065512A5050394079 @default.
- W4386065512 hasAuthorship W4386065512A5056602151 @default.
- W4386065512 hasAuthorship W4386065512A5068360032 @default.
- W4386065512 hasAuthorship W4386065512A5079129211 @default.
- W4386065512 hasAuthorship W4386065512A5085870953 @default.
- W4386065512 hasAuthorship W4386065512A5091825919 @default.
- W4386065512 hasConcept C119857082 @default.
- W4386065512 hasConcept C137293760 @default.
- W4386065512 hasConcept C14036430 @default.
- W4386065512 hasConcept C154945302 @default.
- W4386065512 hasConcept C177212765 @default.
- W4386065512 hasConcept C17744445 @default.
- W4386065512 hasConcept C199360897 @default.
- W4386065512 hasConcept C199539241 @default.
- W4386065512 hasConcept C2524010 @default.
- W4386065512 hasConcept C2988430800 @default.
- W4386065512 hasConcept C33923547 @default.
- W4386065512 hasConcept C41008148 @default.
- W4386065512 hasConcept C43126263 @default.
- W4386065512 hasConcept C77088390 @default.
- W4386065512 hasConcept C78458016 @default.
- W4386065512 hasConcept C86803240 @default.
- W4386065512 hasConcept C99844830 @default.
- W4386065512 hasConceptScore W4386065512C119857082 @default.
- W4386065512 hasConceptScore W4386065512C137293760 @default.
- W4386065512 hasConceptScore W4386065512C14036430 @default.
- W4386065512 hasConceptScore W4386065512C154945302 @default.
- W4386065512 hasConceptScore W4386065512C177212765 @default.
- W4386065512 hasConceptScore W4386065512C17744445 @default.
- W4386065512 hasConceptScore W4386065512C199360897 @default.
- W4386065512 hasConceptScore W4386065512C199539241 @default.
- W4386065512 hasConceptScore W4386065512C2524010 @default.
- W4386065512 hasConceptScore W4386065512C2988430800 @default.
- W4386065512 hasConceptScore W4386065512C33923547 @default.
- W4386065512 hasConceptScore W4386065512C41008148 @default.
- W4386065512 hasConceptScore W4386065512C43126263 @default.
- W4386065512 hasConceptScore W4386065512C77088390 @default.
- W4386065512 hasConceptScore W4386065512C78458016 @default.
- W4386065512 hasConceptScore W4386065512C86803240 @default.
- W4386065512 hasConceptScore W4386065512C99844830 @default.
- W4386065512 hasLocation W43860655121 @default.
- W4386065512 hasOpenAccess W4386065512 @default.
- W4386065512 hasPrimaryLocation W43860655121 @default.
- W4386065512 hasRelatedWork W1978725950 @default.
- W4386065512 hasRelatedWork W1989705153 @default.
- W4386065512 hasRelatedWork W2081035100 @default.
- W4386065512 hasRelatedWork W2497175360 @default.
- W4386065512 hasRelatedWork W2961085424 @default.
- W4386065512 hasRelatedWork W3206324740 @default.
- W4386065512 hasRelatedWork W4286629047 @default.
- W4386065512 hasRelatedWork W4306321456 @default.
- W4386065512 hasRelatedWork W4306674287 @default.
- W4386065512 hasRelatedWork W4224009465 @default.
- W4386065512 isParatext "false" @default.
- W4386065512 isRetracted "false" @default.
- W4386065512 workType "article" @default.