Matches in SemOpenAlex for { <https://semopenalex.org/work/W2998726553> ?p ?o ?g. }
Showing items 1 to 62 of
62
with 100 items per page.
- W2998726553 endingPage "6549" @default.
- W2998726553 startingPage "6542" @default.
- W2998726553 abstract "Model compression has become necessary when applying neural networks (NN) into many real application tasks that can accept slightly-reduced model accuracy but with strict tolerance to model complexity. Recently, Knowledge Distillation, which distills the knowledge from well-trained and highly complex teacher model into a compact student model, has been widely used for model compression. However, under the strict requirement on the resource cost, it is quite challenging to make student model achieve comparable performance with the teacher one, essentially due to the drastically-reduced expressiveness ability of the compact student model. Inspired by the nature of the expressiveness ability in NN, we propose to use multi-segment activation, which can significantly improve the expressiveness ability with very little cost, in the compact student model. Specifically, we propose a highly efficient multi-segment activation, called Light Multi-segment Activation (LMA), which can rapidly produce multiple linear regions with very few parameters by leveraging the statistical information. With using LMA, the compact student model is capable of achieving much better performance effectively and efficiently, than the ReLU-equipped one with same model complexity. Furthermore, the proposed method is compatible with other model compression techniques, such as quantization, which means they can be used jointly for better compression performance. Experiments on state-of-the-art NN architectures over the real-world tasks demonstrate the effectiveness and extensibility of the LMA." @default.
- W2998726553 created "2020-01-10" @default.
- W2998726553 creator A5000535032 @default.
- W2998726553 creator A5021438219 @default.
- W2998726553 creator A5061200287 @default.
- W2998726553 creator A5070990160 @default.
- W2998726553 creator A5074453054 @default.
- W2998726553 date "2020-04-03" @default.
- W2998726553 modified "2023-09-23" @default.
- W2998726553 title "Light Multi-Segment Activation for Model Compression" @default.
- W2998726553 doi "https://doi.org/10.1609/aaai.v34i04.6128" @default.
- W2998726553 hasPublicationYear "2020" @default.
- W2998726553 type Work @default.
- W2998726553 sameAs 2998726553 @default.
- W2998726553 citedByCount "0" @default.
- W2998726553 crossrefType "journal-article" @default.
- W2998726553 hasAuthorship W2998726553A5000535032 @default.
- W2998726553 hasAuthorship W2998726553A5021438219 @default.
- W2998726553 hasAuthorship W2998726553A5061200287 @default.
- W2998726553 hasAuthorship W2998726553A5070990160 @default.
- W2998726553 hasAuthorship W2998726553A5074453054 @default.
- W2998726553 hasBestOaLocation W29987265531 @default.
- W2998726553 hasConcept C11413529 @default.
- W2998726553 hasConcept C154945302 @default.
- W2998726553 hasConcept C159985019 @default.
- W2998726553 hasConcept C180016635 @default.
- W2998726553 hasConcept C192562407 @default.
- W2998726553 hasConcept C199360897 @default.
- W2998726553 hasConcept C28855332 @default.
- W2998726553 hasConcept C32833848 @default.
- W2998726553 hasConcept C41008148 @default.
- W2998726553 hasConceptScore W2998726553C11413529 @default.
- W2998726553 hasConceptScore W2998726553C154945302 @default.
- W2998726553 hasConceptScore W2998726553C159985019 @default.
- W2998726553 hasConceptScore W2998726553C180016635 @default.
- W2998726553 hasConceptScore W2998726553C192562407 @default.
- W2998726553 hasConceptScore W2998726553C199360897 @default.
- W2998726553 hasConceptScore W2998726553C28855332 @default.
- W2998726553 hasConceptScore W2998726553C32833848 @default.
- W2998726553 hasConceptScore W2998726553C41008148 @default.
- W2998726553 hasIssue "04" @default.
- W2998726553 hasLocation W29987265531 @default.
- W2998726553 hasLocation W29987265532 @default.
- W2998726553 hasOpenAccess W2998726553 @default.
- W2998726553 hasPrimaryLocation W29987265531 @default.
- W2998726553 hasRelatedWork W1984116007 @default.
- W2998726553 hasRelatedWork W2051744418 @default.
- W2998726553 hasRelatedWork W2165124476 @default.
- W2998726553 hasRelatedWork W2362760518 @default.
- W2998726553 hasRelatedWork W2364420803 @default.
- W2998726553 hasRelatedWork W2368425793 @default.
- W2998726553 hasRelatedWork W2372886617 @default.
- W2998726553 hasRelatedWork W25068511 @default.
- W2998726553 hasRelatedWork W2998852367 @default.
- W2998726553 hasRelatedWork W3013297713 @default.
- W2998726553 hasVolume "34" @default.
- W2998726553 isParatext "false" @default.
- W2998726553 isRetracted "false" @default.
- W2998726553 magId "2998726553" @default.
- W2998726553 workType "article" @default.