Matches in SemOpenAlex for { <https://semopenalex.org/work/W2903396356> ?p ?o ?g. }
- W2903396356 endingPage "786" @default.
- W2903396356 startingPage "775" @default.
- W2903396356 abstract "Knowledge distillation (KD) aims to train a lightweight classifier suitable to provide accurate inference with constrained resources in multi-label learning. Instead of directly consuming feature-label pairs, the classifier is trained by a teacher, i.e., a high-capacity model whose training may be resource-hungry. The accuracy of the classifier trained this way is usually suboptimal because it is difficult to learn the true data distribution from the teacher. An alternative method is to adversarially train the classifier against a discriminator in a two-player game akin to generative adversarial networks (GAN), which can ensure the classifier to learn the true data distribution at the equilibrium of this game. However, it may take excessively long time for such a two-player game to reach equilibrium due to high-variance gradient updates. To address these limitations, we propose a three-player game named KDGAN consisting of a classifier, a teacher, and a discriminator. The classifier and the teacher learn from each other via distillation losses and are adversarially trained against the discriminator via adversarial losses. By simultaneously optimizing the distillation and adversarial losses, the classifier will learn the true data distribution at the equilibrium. We approximate the discrete distribution learned by the classifier (or the teacher) with a concrete distribution. From the concrete distribution, we generate continuous samples to obtain low-variance gradient updates, which speed up the training. Extensive experiments using real datasets confirm the superiority of KDGAN in both accuracy and training speed." @default.
- W2903396356 created "2018-12-11" @default.
- W2903396356 creator A5021379796 @default.
- W2903396356 creator A5022290876 @default.
- W2903396356 creator A5065037360 @default.
- W2903396356 creator A5068936523 @default.
- W2903396356 date "2018-01-01" @default.
- W2903396356 modified "2023-09-27" @default.
- W2903396356 title "KDGAN: Knowledge Distillation with Generative Adversarial Networks" @default.
- W2903396356 cites W1526445072 @default.
- W2903396356 cites W1686810756 @default.
- W2903396356 cites W1821462560 @default.
- W2903396356 cites W1933349210 @default.
- W2903396356 cites W1983988843 @default.
- W2903396356 cites W2056624866 @default.
- W2903396356 cites W2064675550 @default.
- W2903396356 cites W2087560337 @default.
- W2903396356 cites W2099471712 @default.
- W2903396356 cites W2108598243 @default.
- W2903396356 cites W2112796928 @default.
- W2903396356 cites W2114315281 @default.
- W2903396356 cites W2134797427 @default.
- W2903396356 cites W2141225381 @default.
- W2903396356 cites W2153579005 @default.
- W2903396356 cites W2173379916 @default.
- W2903396356 cites W2194775991 @default.
- W2903396356 cites W2250384498 @default.
- W2903396356 cites W2294370754 @default.
- W2903396356 cites W2296701362 @default.
- W2903396356 cites W2335875860 @default.
- W2903396356 cites W2514480375 @default.
- W2903396356 cites W2521028896 @default.
- W2903396356 cites W2536305071 @default.
- W2903396356 cites W2547875792 @default.
- W2903396356 cites W2554314924 @default.
- W2903396356 cites W2596763562 @default.
- W2903396356 cites W2602076750 @default.
- W2903396356 cites W2746626573 @default.
- W2903396356 cites W2766736793 @default.
- W2903396356 cites W2787017828 @default.
- W2903396356 cites W2952165242 @default.
- W2903396356 cites W2952745707 @default.
- W2903396356 cites W2962946266 @default.
- W2903396356 cites W2963373786 @default.
- W2903396356 cites W2963474063 @default.
- W2903396356 cites W2964017345 @default.
- W2903396356 cites W2964201867 @default.
- W2903396356 cites W2964268978 @default.
- W2903396356 cites W3101023724 @default.
- W2903396356 cites W3118608800 @default.
- W2903396356 cites W753847829 @default.
- W2903396356 cites W3023820860 @default.
- W2903396356 hasPublicationYear "2018" @default.
- W2903396356 type Work @default.
- W2903396356 sameAs 2903396356 @default.
- W2903396356 citedByCount "49" @default.
- W2903396356 countsByYear W29033963562019 @default.
- W2903396356 countsByYear W29033963562020 @default.
- W2903396356 countsByYear W29033963562021 @default.
- W2903396356 countsByYear W29033963562022 @default.
- W2903396356 crossrefType "proceedings-article" @default.
- W2903396356 hasAuthorship W2903396356A5021379796 @default.
- W2903396356 hasAuthorship W2903396356A5022290876 @default.
- W2903396356 hasAuthorship W2903396356A5065037360 @default.
- W2903396356 hasAuthorship W2903396356A5068936523 @default.
- W2903396356 hasConcept C119857082 @default.
- W2903396356 hasConcept C153180895 @default.
- W2903396356 hasConcept C154945302 @default.
- W2903396356 hasConcept C178790620 @default.
- W2903396356 hasConcept C185592680 @default.
- W2903396356 hasConcept C204030448 @default.
- W2903396356 hasConcept C2776214188 @default.
- W2903396356 hasConcept C2779803651 @default.
- W2903396356 hasConcept C37736160 @default.
- W2903396356 hasConcept C39890363 @default.
- W2903396356 hasConcept C41008148 @default.
- W2903396356 hasConcept C76155785 @default.
- W2903396356 hasConcept C94915269 @default.
- W2903396356 hasConcept C95623464 @default.
- W2903396356 hasConceptScore W2903396356C119857082 @default.
- W2903396356 hasConceptScore W2903396356C153180895 @default.
- W2903396356 hasConceptScore W2903396356C154945302 @default.
- W2903396356 hasConceptScore W2903396356C178790620 @default.
- W2903396356 hasConceptScore W2903396356C185592680 @default.
- W2903396356 hasConceptScore W2903396356C204030448 @default.
- W2903396356 hasConceptScore W2903396356C2776214188 @default.
- W2903396356 hasConceptScore W2903396356C2779803651 @default.
- W2903396356 hasConceptScore W2903396356C37736160 @default.
- W2903396356 hasConceptScore W2903396356C39890363 @default.
- W2903396356 hasConceptScore W2903396356C41008148 @default.
- W2903396356 hasConceptScore W2903396356C76155785 @default.
- W2903396356 hasConceptScore W2903396356C94915269 @default.
- W2903396356 hasConceptScore W2903396356C95623464 @default.
- W2903396356 hasLocation W29033963561 @default.
- W2903396356 hasOpenAccess W2903396356 @default.
- W2903396356 hasPrimaryLocation W29033963561 @default.
- W2903396356 hasRelatedWork W1821462560 @default.
- W2903396356 hasRelatedWork W2099471712 @default.