Matches in SemOpenAlex for { <https://semopenalex.org/work/W4367000542> ?p ?o ?g. }
Showing items 1 to 69 of
69
with 100 items per page.
- W4367000542 abstract "We address the challenge of getting efficient yet accurate recognition systems with limited labels. While recognition models improve with model size and amount of data, many specialized applications of computer vision have severe resource constraints both during training and inference. Transfer learning is an effective solution for training with few labels, however often at the expense of a computationally costly fine-tuning of large base models. We propose to mitigate this unpleasant trade-off between compute and accuracy via semi-supervised cross-domain distillation from a set of diverse source models. Initially, we show how to use task similarity metrics to select a single suitable source model to distill from, and that a good selection process is imperative for good downstream performance of a target model. We dub this approach DistillNearest. Though effective, DistillNearest assumes a single source model matches the target task, which is not always the case. To alleviate this, we propose a weighted multi-source distillation method to distill multiple source models trained on different domains weighted by their relevance for the target task into a single efficient model (named DistillWeighted). Our methods need no access to source data, and merely need features and pseudo-labels of the source models. When the goal is accurate recognition under computational constraints, both DistillNearest and DistillWeighted approaches outperform both transfer learning from strong ImageNet initializations as well as state-of-the-art semi-supervised techniques such as FixMatch. Averaged over 8 diverse target tasks our multi-source method outperforms the baselines by 5.6%-points and 4.5%-points, respectively." @default.
- W4367000542 created "2023-04-27" @default.
- W4367000542 creator A5016682510 @default.
- W4367000542 creator A5019098232 @default.
- W4367000542 creator A5036548506 @default.
- W4367000542 date "2023-04-24" @default.
- W4367000542 modified "2023-10-01" @default.
- W4367000542 title "Distilling from Similar Tasks for Transfer Learning on a Budget" @default.
- W4367000542 doi "https://doi.org/10.48550/arxiv.2304.12314" @default.
- W4367000542 hasPublicationYear "2023" @default.
- W4367000542 type Work @default.
- W4367000542 citedByCount "0" @default.
- W4367000542 crossrefType "posted-content" @default.
- W4367000542 hasAuthorship W4367000542A5016682510 @default.
- W4367000542 hasAuthorship W4367000542A5019098232 @default.
- W4367000542 hasAuthorship W4367000542A5036548506 @default.
- W4367000542 hasBestOaLocation W43670005421 @default.
- W4367000542 hasConcept C103278499 @default.
- W4367000542 hasConcept C111919701 @default.
- W4367000542 hasConcept C115961682 @default.
- W4367000542 hasConcept C119857082 @default.
- W4367000542 hasConcept C124101348 @default.
- W4367000542 hasConcept C150899416 @default.
- W4367000542 hasConcept C154945302 @default.
- W4367000542 hasConcept C158154518 @default.
- W4367000542 hasConcept C162324750 @default.
- W4367000542 hasConcept C177264268 @default.
- W4367000542 hasConcept C17744445 @default.
- W4367000542 hasConcept C187736073 @default.
- W4367000542 hasConcept C199360897 @default.
- W4367000542 hasConcept C199539241 @default.
- W4367000542 hasConcept C2776214188 @default.
- W4367000542 hasConcept C2780451532 @default.
- W4367000542 hasConcept C41008148 @default.
- W4367000542 hasConcept C98045186 @default.
- W4367000542 hasConceptScore W4367000542C103278499 @default.
- W4367000542 hasConceptScore W4367000542C111919701 @default.
- W4367000542 hasConceptScore W4367000542C115961682 @default.
- W4367000542 hasConceptScore W4367000542C119857082 @default.
- W4367000542 hasConceptScore W4367000542C124101348 @default.
- W4367000542 hasConceptScore W4367000542C150899416 @default.
- W4367000542 hasConceptScore W4367000542C154945302 @default.
- W4367000542 hasConceptScore W4367000542C158154518 @default.
- W4367000542 hasConceptScore W4367000542C162324750 @default.
- W4367000542 hasConceptScore W4367000542C177264268 @default.
- W4367000542 hasConceptScore W4367000542C17744445 @default.
- W4367000542 hasConceptScore W4367000542C187736073 @default.
- W4367000542 hasConceptScore W4367000542C199360897 @default.
- W4367000542 hasConceptScore W4367000542C199539241 @default.
- W4367000542 hasConceptScore W4367000542C2776214188 @default.
- W4367000542 hasConceptScore W4367000542C2780451532 @default.
- W4367000542 hasConceptScore W4367000542C41008148 @default.
- W4367000542 hasConceptScore W4367000542C98045186 @default.
- W4367000542 hasLocation W43670005421 @default.
- W4367000542 hasOpenAccess W4367000542 @default.
- W4367000542 hasPrimaryLocation W43670005421 @default.
- W4367000542 hasRelatedWork W1509467138 @default.
- W4367000542 hasRelatedWork W2036947891 @default.
- W4367000542 hasRelatedWork W2365088826 @default.
- W4367000542 hasRelatedWork W2413863797 @default.
- W4367000542 hasRelatedWork W2912887033 @default.
- W4367000542 hasRelatedWork W2960456850 @default.
- W4367000542 hasRelatedWork W4281382123 @default.
- W4367000542 hasRelatedWork W4308262314 @default.
- W4367000542 hasRelatedWork W4318834068 @default.
- W4367000542 hasRelatedWork W293603395 @default.
- W4367000542 isParatext "false" @default.
- W4367000542 isRetracted "false" @default.
- W4367000542 workType "article" @default.