Matches in SemOpenAlex for { <https://semopenalex.org/work/W2802862615> ?p ?o ?g. }
Showing items 1 to 84 of
84
with 100 items per page.
- W2802862615 abstract "Activation functions provide deep neural networks the non-linearity that is necessary to learn complex distributions. It is still inconclusive what is the optimal shape for the activation function. In this work, we introduce a novel type of activation function of which the shape is learned with network training. The proposed Look-up Table Unit (LuTU) stores a set of anchor points in a look-up table like structure, and the activation function is generated from the anchor points by either linear interpolation or smoothing with a single period cosine mask function. LuTU is in theory able to approximate any univariate function. By observing the learned shapes of LuTU, we further propose a Mixture of Gaussian Unit (MoGU) that can learn similar non-linear shapes with much fewer parameters. Finally, we use a multiple activation function fusion framework that combines multiple types of functions to achieve better performance. The inference complexity of multiple activation function fusion is constant with linear interpolation approximation. Our experiments on a synthetic dataset, ImageNet, and CIFAR-10 demonstrate that the proposed method outperforms traditional ReLU family activation functions. On the ImageNet dataset, our method achieves 1.47% and 1.0% higher accuracy on ResNet-18 and ResNet-34 models, respectively. With the proposed activation function, we can design a network that has the same performance as ResNet-34 but 8 fewer convolutional layers." @default.
- W2802862615 created "2018-05-17" @default.
- W2802862615 creator A5012176612 @default.
- W2802862615 creator A5020007852 @default.
- W2802862615 creator A5076117344 @default.
- W2802862615 date "2018-03-01" @default.
- W2802862615 modified "2023-10-16" @default.
- W2802862615 title "Look-Up Table Unit Activation Function for Deep Convolutional Neural Networks" @default.
- W2802862615 cites W1677182931 @default.
- W2802862615 cites W2117539524 @default.
- W2802862615 cites W2139212933 @default.
- W2802862615 cites W2194775991 @default.
- W2802862615 cites W2336829316 @default.
- W2802862615 cites W2963446712 @default.
- W2802862615 doi "https://doi.org/10.1109/wacv.2018.00139" @default.
- W2802862615 hasPublicationYear "2018" @default.
- W2802862615 type Work @default.
- W2802862615 sameAs 2802862615 @default.
- W2802862615 citedByCount "11" @default.
- W2802862615 countsByYear W28028626152018 @default.
- W2802862615 countsByYear W28028626152019 @default.
- W2802862615 countsByYear W28028626152020 @default.
- W2802862615 countsByYear W28028626152021 @default.
- W2802862615 countsByYear W28028626152022 @default.
- W2802862615 countsByYear W28028626152023 @default.
- W2802862615 crossrefType "proceedings-article" @default.
- W2802862615 hasAuthorship W2802862615A5012176612 @default.
- W2802862615 hasAuthorship W2802862615A5020007852 @default.
- W2802862615 hasAuthorship W2802862615A5076117344 @default.
- W2802862615 hasConcept C104114177 @default.
- W2802862615 hasConcept C11413529 @default.
- W2802862615 hasConcept C119857082 @default.
- W2802862615 hasConcept C137800194 @default.
- W2802862615 hasConcept C14036430 @default.
- W2802862615 hasConcept C153180895 @default.
- W2802862615 hasConcept C154945302 @default.
- W2802862615 hasConcept C161584116 @default.
- W2802862615 hasConcept C171836373 @default.
- W2802862615 hasConcept C199163554 @default.
- W2802862615 hasConcept C2776214188 @default.
- W2802862615 hasConcept C31972630 @default.
- W2802862615 hasConcept C3770464 @default.
- W2802862615 hasConcept C38365724 @default.
- W2802862615 hasConcept C41008148 @default.
- W2802862615 hasConcept C50644808 @default.
- W2802862615 hasConcept C78458016 @default.
- W2802862615 hasConcept C81363708 @default.
- W2802862615 hasConcept C86803240 @default.
- W2802862615 hasConceptScore W2802862615C104114177 @default.
- W2802862615 hasConceptScore W2802862615C11413529 @default.
- W2802862615 hasConceptScore W2802862615C119857082 @default.
- W2802862615 hasConceptScore W2802862615C137800194 @default.
- W2802862615 hasConceptScore W2802862615C14036430 @default.
- W2802862615 hasConceptScore W2802862615C153180895 @default.
- W2802862615 hasConceptScore W2802862615C154945302 @default.
- W2802862615 hasConceptScore W2802862615C161584116 @default.
- W2802862615 hasConceptScore W2802862615C171836373 @default.
- W2802862615 hasConceptScore W2802862615C199163554 @default.
- W2802862615 hasConceptScore W2802862615C2776214188 @default.
- W2802862615 hasConceptScore W2802862615C31972630 @default.
- W2802862615 hasConceptScore W2802862615C3770464 @default.
- W2802862615 hasConceptScore W2802862615C38365724 @default.
- W2802862615 hasConceptScore W2802862615C41008148 @default.
- W2802862615 hasConceptScore W2802862615C50644808 @default.
- W2802862615 hasConceptScore W2802862615C78458016 @default.
- W2802862615 hasConceptScore W2802862615C81363708 @default.
- W2802862615 hasConceptScore W2802862615C86803240 @default.
- W2802862615 hasLocation W28028626151 @default.
- W2802862615 hasOpenAccess W2802862615 @default.
- W2802862615 hasPrimaryLocation W28028626151 @default.
- W2802862615 hasRelatedWork W1542971759 @default.
- W2802862615 hasRelatedWork W1981806781 @default.
- W2802862615 hasRelatedWork W2099531806 @default.
- W2802862615 hasRelatedWork W2360727993 @default.
- W2802862615 hasRelatedWork W2370429033 @default.
- W2802862615 hasRelatedWork W2767651786 @default.
- W2802862615 hasRelatedWork W2886958150 @default.
- W2802862615 hasRelatedWork W2912288872 @default.
- W2802862615 hasRelatedWork W3100128465 @default.
- W2802862615 hasRelatedWork W564581980 @default.
- W2802862615 isParatext "false" @default.
- W2802862615 isRetracted "false" @default.
- W2802862615 magId "2802862615" @default.
- W2802862615 workType "article" @default.