Matches in SemOpenAlex for { <https://semopenalex.org/work/W4385060903> ?p ?o ?g. }
Showing items 1 to 96 of
96
with 100 items per page.
- W4385060903 endingPage "0" @default.
- W4385060903 startingPage "0" @default.
- W4385060903 abstract "Activation functions(AFs) are crucial components in neural networks for deep learning. Piecewise linear functions(PLFs) have been widely employed to act as AFs, thanks to their computational efficiencies and simplicities. However, PLFs are not completely differentiable, have potential problems of training. The analytical expressions of AFs based on pure PLFs can be smoothed via mollified square root function(MSRF) method, inspired by SquarePlus method of ReLU approximation. In this paper, we propose a proposition defining AFs as maximum or minimum of two PLFs, and transform the results into a smoothed function via MSRF method. Based on MSRF, we modify the well-known AFs composed of two PLFs, three or four PLFs to regularized ones systematically, including ReLU, LReLU, vReLU, Step, Biplolar, BReLU(Bounded ReLU), Htanh(Hard Tanh), Pan(Frying pan function), STF(Soft Thresholding Formulas), HTF(Soft Thresholding Formulas), SReLU(S-shaped ReLU), MReLU(Mexican hat type ReLU), TSF(Trapezoid-shaped function) functions. Additionally, according to the equivalences of SquarePlus and SoftPlus functions, some classic compound AFs, such as ELU, Swish, Mish, SoftSign Logish and DLU functions, can be expressed via MSRF method also. The derivatives of their mollified versions demonstrate their smoothness properties. The proposed method can be extended to AFs composed of multiple PLFs easily, which will be investigated deeply in the future." @default.
- W4385060903 created "2023-07-23" @default.
- W4385060903 creator A5007977858 @default.
- W4385060903 creator A5030786536 @default.
- W4385060903 creator A5053839400 @default.
- W4385060903 creator A5060587784 @default.
- W4385060903 date "2023-01-01" @default.
- W4385060903 modified "2023-10-18" @default.
- W4385060903 title "Smoothing piecewise linear activation functions based on mollified square root functions" @default.
- W4385060903 cites W2103496339 @default.
- W4385060903 cites W2137983211 @default.
- W4385060903 cites W2159152281 @default.
- W4385060903 cites W2512630954 @default.
- W4385060903 cites W2765421612 @default.
- W4385060903 cites W3023212902 @default.
- W4385060903 cites W3089733125 @default.
- W4385060903 cites W3100321043 @default.
- W4385060903 cites W3133902371 @default.
- W4385060903 cites W3141520081 @default.
- W4385060903 cites W3174889011 @default.
- W4385060903 cites W4205694016 @default.
- W4385060903 cites W4226455337 @default.
- W4385060903 cites W4281719784 @default.
- W4385060903 cites W4283791586 @default.
- W4385060903 cites W4292313505 @default.
- W4385060903 cites W4296234188 @default.
- W4385060903 cites W4310253114 @default.
- W4385060903 cites W4324145582 @default.
- W4385060903 doi "https://doi.org/10.3934/mfc.2023032" @default.
- W4385060903 hasPublicationYear "2023" @default.
- W4385060903 type Work @default.
- W4385060903 citedByCount "0" @default.
- W4385060903 crossrefType "journal-article" @default.
- W4385060903 hasAuthorship W4385060903A5007977858 @default.
- W4385060903 hasAuthorship W4385060903A5030786536 @default.
- W4385060903 hasAuthorship W4385060903A5053839400 @default.
- W4385060903 hasAuthorship W4385060903A5060587784 @default.
- W4385060903 hasBestOaLocation W43850609031 @default.
- W4385060903 hasConcept C102634674 @default.
- W4385060903 hasConcept C11413529 @default.
- W4385060903 hasConcept C11577676 @default.
- W4385060903 hasConcept C115961682 @default.
- W4385060903 hasConcept C134306372 @default.
- W4385060903 hasConcept C14036430 @default.
- W4385060903 hasConcept C154945302 @default.
- W4385060903 hasConcept C15627037 @default.
- W4385060903 hasConcept C164660894 @default.
- W4385060903 hasConcept C17095337 @default.
- W4385060903 hasConcept C191178318 @default.
- W4385060903 hasConcept C202444582 @default.
- W4385060903 hasConcept C202615002 @default.
- W4385060903 hasConcept C2524010 @default.
- W4385060903 hasConcept C28826006 @default.
- W4385060903 hasConcept C33923547 @default.
- W4385060903 hasConcept C41008148 @default.
- W4385060903 hasConcept C78458016 @default.
- W4385060903 hasConcept C86803240 @default.
- W4385060903 hasConceptScore W4385060903C102634674 @default.
- W4385060903 hasConceptScore W4385060903C11413529 @default.
- W4385060903 hasConceptScore W4385060903C11577676 @default.
- W4385060903 hasConceptScore W4385060903C115961682 @default.
- W4385060903 hasConceptScore W4385060903C134306372 @default.
- W4385060903 hasConceptScore W4385060903C14036430 @default.
- W4385060903 hasConceptScore W4385060903C154945302 @default.
- W4385060903 hasConceptScore W4385060903C15627037 @default.
- W4385060903 hasConceptScore W4385060903C164660894 @default.
- W4385060903 hasConceptScore W4385060903C17095337 @default.
- W4385060903 hasConceptScore W4385060903C191178318 @default.
- W4385060903 hasConceptScore W4385060903C202444582 @default.
- W4385060903 hasConceptScore W4385060903C202615002 @default.
- W4385060903 hasConceptScore W4385060903C2524010 @default.
- W4385060903 hasConceptScore W4385060903C28826006 @default.
- W4385060903 hasConceptScore W4385060903C33923547 @default.
- W4385060903 hasConceptScore W4385060903C41008148 @default.
- W4385060903 hasConceptScore W4385060903C78458016 @default.
- W4385060903 hasConceptScore W4385060903C86803240 @default.
- W4385060903 hasIssue "0" @default.
- W4385060903 hasLocation W43850609031 @default.
- W4385060903 hasOpenAccess W4385060903 @default.
- W4385060903 hasPrimaryLocation W43850609031 @default.
- W4385060903 hasRelatedWork W1980455870 @default.
- W4385060903 hasRelatedWork W1996771141 @default.
- W4385060903 hasRelatedWork W2011767324 @default.
- W4385060903 hasRelatedWork W2017382275 @default.
- W4385060903 hasRelatedWork W2094479588 @default.
- W4385060903 hasRelatedWork W2106956798 @default.
- W4385060903 hasRelatedWork W2132110236 @default.
- W4385060903 hasRelatedWork W2522028554 @default.
- W4385060903 hasRelatedWork W2546087828 @default.
- W4385060903 hasRelatedWork W4301837952 @default.
- W4385060903 hasVolume "0" @default.
- W4385060903 isParatext "false" @default.
- W4385060903 isRetracted "false" @default.
- W4385060903 workType "article" @default.