Matches in SemOpenAlex for { <https://semopenalex.org/work/W4288804572> ?p ?o ?g. }
Showing items 1 to 57 of
57
with 100 items per page.
- W4288804572 abstract "In recent years novel activation functions have been proposed to improve the performance of neural networks, and they show superior performance compared to the ReLU counterpart. However, there are environments, where the availability of complex activations is limited, and usually only the ReLU is supported. In this paper we propose methods that can be used to improve the performance of ReLU networks by using these efficient novel activations during model training. More specifically, we propose ensemble activations that are composed of the ReLU and one of these novel activations. Furthermore, the coefficients of the ensemble are neither fixed nor learned, but are progressively updated during the training process in a way that by the end of the training only the ReLU activations remain active in the network and the other activations can be removed. This means that in inference time the network contains ReLU activations only. We perform extensive evaluations on the ImageNet classification task using various compact network architectures and various novel activation functions. Results show 0.2-0.8% top-1 accuracy gain, which confirms the applicability of the proposed methods. Furthermore, we demonstrate the proposed methods on semantic segmentation and we boost the performance of a compact segmentation network by 0.34% mIOU on the Cityscapes dataset." @default.
- W4288804572 created "2022-07-30" @default.
- W4288804572 creator A5056300623 @default.
- W4288804572 date "2022-07-28" @default.
- W4288804572 modified "2023-09-29" @default.
- W4288804572 title "PEA: Improving the Performance of ReLU Networks for Free by Using Progressive Ensemble Activations" @default.
- W4288804572 doi "https://doi.org/10.48550/arxiv.2207.14074" @default.
- W4288804572 hasPublicationYear "2022" @default.
- W4288804572 type Work @default.
- W4288804572 citedByCount "0" @default.
- W4288804572 crossrefType "posted-content" @default.
- W4288804572 hasAuthorship W4288804572A5056300623 @default.
- W4288804572 hasBestOaLocation W42888045721 @default.
- W4288804572 hasConcept C111919701 @default.
- W4288804572 hasConcept C119857082 @default.
- W4288804572 hasConcept C153180895 @default.
- W4288804572 hasConcept C154945302 @default.
- W4288804572 hasConcept C162324750 @default.
- W4288804572 hasConcept C187736073 @default.
- W4288804572 hasConcept C21547014 @default.
- W4288804572 hasConcept C2776214188 @default.
- W4288804572 hasConcept C2778915421 @default.
- W4288804572 hasConcept C2780451532 @default.
- W4288804572 hasConcept C41008148 @default.
- W4288804572 hasConcept C50644808 @default.
- W4288804572 hasConcept C89600930 @default.
- W4288804572 hasConcept C98045186 @default.
- W4288804572 hasConceptScore W4288804572C111919701 @default.
- W4288804572 hasConceptScore W4288804572C119857082 @default.
- W4288804572 hasConceptScore W4288804572C153180895 @default.
- W4288804572 hasConceptScore W4288804572C154945302 @default.
- W4288804572 hasConceptScore W4288804572C162324750 @default.
- W4288804572 hasConceptScore W4288804572C187736073 @default.
- W4288804572 hasConceptScore W4288804572C21547014 @default.
- W4288804572 hasConceptScore W4288804572C2776214188 @default.
- W4288804572 hasConceptScore W4288804572C2778915421 @default.
- W4288804572 hasConceptScore W4288804572C2780451532 @default.
- W4288804572 hasConceptScore W4288804572C41008148 @default.
- W4288804572 hasConceptScore W4288804572C50644808 @default.
- W4288804572 hasConceptScore W4288804572C89600930 @default.
- W4288804572 hasConceptScore W4288804572C98045186 @default.
- W4288804572 hasLocation W42888045721 @default.
- W4288804572 hasOpenAccess W4288804572 @default.
- W4288804572 hasPrimaryLocation W42888045721 @default.
- W4288804572 hasRelatedWork W12703013 @default.
- W4288804572 hasRelatedWork W1383942 @default.
- W4288804572 hasRelatedWork W14128562 @default.
- W4288804572 hasRelatedWork W14516383 @default.
- W4288804572 hasRelatedWork W1850987 @default.
- W4288804572 hasRelatedWork W4085024 @default.
- W4288804572 hasRelatedWork W4412456 @default.
- W4288804572 hasRelatedWork W4771408 @default.
- W4288804572 hasRelatedWork W592020 @default.
- W4288804572 hasRelatedWork W9321062 @default.
- W4288804572 isParatext "false" @default.
- W4288804572 isRetracted "false" @default.
- W4288804572 workType "article" @default.