Matches in SemOpenAlex for { <https://semopenalex.org/work/W3204133379> ?p ?o ?g. }
Showing items 1 to 75 of
75
with 100 items per page.
- W3204133379 abstract "The choice of activation functions is crucial for modern deep neural networks. Popular hand-designed activation functions like Rectified Linear Unit(ReLU) and its variants show promising performance in various tasks and models. Swish, the automatically discovered activation function, has been proposed and outperforms ReLU on many challenging datasets. However, it has two main drawbacks. First, the tree-based search space is highly discrete and restricted, which is difficult for searching. Second, the sample-based searching method is inefficient, making it infeasible to find specialized activation functions for each dataset or neural architecture. To tackle these drawbacks, we propose a new activation function called Piecewise Linear Unit(PWLU), which incorporates a carefully designed formulation and learning method. It can learn specialized activation functions and achieves SOTA performance on large-scale datasets like ImageNet and COCO. For example, on ImageNet classification dataset, PWLU improves 0.9%/0.53%/1.0%/1.7%/1.0% top-1 accuracy over Swish for ResNet-18/ResNet-50/MobileNet-V2/MobileNet-V3/EfficientNet-B0. PWLU is also easy to implement and efficient at inference, which can be widely applied in real-world applications." @default.
- W3204133379 created "2021-10-11" @default.
- W3204133379 creator A5006894022 @default.
- W3204133379 creator A5007052694 @default.
- W3204133379 creator A5047316686 @default.
- W3204133379 date "2021-04-08" @default.
- W3204133379 modified "2023-09-27" @default.
- W3204133379 title "Learning specialized activation functions with the Piecewise Linear Unit" @default.
- W3204133379 doi "https://doi.org/10.48550/arxiv.2104.03693" @default.
- W3204133379 hasPublicationYear "2021" @default.
- W3204133379 type Work @default.
- W3204133379 sameAs 3204133379 @default.
- W3204133379 citedByCount "0" @default.
- W3204133379 crossrefType "posted-content" @default.
- W3204133379 hasAuthorship W3204133379A5006894022 @default.
- W3204133379 hasAuthorship W3204133379A5007052694 @default.
- W3204133379 hasAuthorship W3204133379A5047316686 @default.
- W3204133379 hasBestOaLocation W32041333791 @default.
- W3204133379 hasConcept C113174947 @default.
- W3204133379 hasConcept C119857082 @default.
- W3204133379 hasConcept C121332964 @default.
- W3204133379 hasConcept C134306372 @default.
- W3204133379 hasConcept C14036430 @default.
- W3204133379 hasConcept C154945302 @default.
- W3204133379 hasConcept C164660894 @default.
- W3204133379 hasConcept C17095337 @default.
- W3204133379 hasConcept C2524010 @default.
- W3204133379 hasConcept C2776214188 @default.
- W3204133379 hasConcept C2778755073 @default.
- W3204133379 hasConcept C2944601119 @default.
- W3204133379 hasConcept C2984842247 @default.
- W3204133379 hasConcept C33923547 @default.
- W3204133379 hasConcept C38365724 @default.
- W3204133379 hasConcept C41008148 @default.
- W3204133379 hasConcept C50644808 @default.
- W3204133379 hasConcept C62520636 @default.
- W3204133379 hasConcept C78458016 @default.
- W3204133379 hasConcept C86803240 @default.
- W3204133379 hasConceptScore W3204133379C113174947 @default.
- W3204133379 hasConceptScore W3204133379C119857082 @default.
- W3204133379 hasConceptScore W3204133379C121332964 @default.
- W3204133379 hasConceptScore W3204133379C134306372 @default.
- W3204133379 hasConceptScore W3204133379C14036430 @default.
- W3204133379 hasConceptScore W3204133379C154945302 @default.
- W3204133379 hasConceptScore W3204133379C164660894 @default.
- W3204133379 hasConceptScore W3204133379C17095337 @default.
- W3204133379 hasConceptScore W3204133379C2524010 @default.
- W3204133379 hasConceptScore W3204133379C2776214188 @default.
- W3204133379 hasConceptScore W3204133379C2778755073 @default.
- W3204133379 hasConceptScore W3204133379C2944601119 @default.
- W3204133379 hasConceptScore W3204133379C2984842247 @default.
- W3204133379 hasConceptScore W3204133379C33923547 @default.
- W3204133379 hasConceptScore W3204133379C38365724 @default.
- W3204133379 hasConceptScore W3204133379C41008148 @default.
- W3204133379 hasConceptScore W3204133379C50644808 @default.
- W3204133379 hasConceptScore W3204133379C62520636 @default.
- W3204133379 hasConceptScore W3204133379C78458016 @default.
- W3204133379 hasConceptScore W3204133379C86803240 @default.
- W3204133379 hasLocation W32041333791 @default.
- W3204133379 hasOpenAccess W3204133379 @default.
- W3204133379 hasPrimaryLocation W32041333791 @default.
- W3204133379 hasRelatedWork W2008505711 @default.
- W3204133379 hasRelatedWork W2511279186 @default.
- W3204133379 hasRelatedWork W2893539081 @default.
- W3204133379 hasRelatedWork W3005627584 @default.
- W3204133379 hasRelatedWork W3088445005 @default.
- W3204133379 hasRelatedWork W3129882263 @default.
- W3204133379 hasRelatedWork W4302084508 @default.
- W3204133379 hasRelatedWork W4306794901 @default.
- W3204133379 hasRelatedWork W1629725936 @default.
- W3204133379 hasRelatedWork W4287869072 @default.
- W3204133379 isParatext "false" @default.
- W3204133379 isRetracted "false" @default.
- W3204133379 magId "3204133379" @default.
- W3204133379 workType "article" @default.