Matches in SemOpenAlex for { <https://semopenalex.org/work/W4383501712> ?p ?o ?g. }
Showing items 1 to 81 of
81
with 100 items per page.
- W4383501712 abstract "The activation function is one of key elements in modern machine learning algorithms. However, some broadly-used activation functions are exceptionally complex, e.g., GELU in Transformer-based algorithms, which makes their precise yet efficient VLSI implementations extremely hard. In this paper, two series of hardware-friendly activation function designs, DNR and PWL, and their VLSI implementations are proposed. Both are specifically designed to replace GELU, which is widely used in Transformer-related applications. Instead of utilizing traditional lookup-table (LUT)-based approximation methods, this paper introduces new activation functions that are not only hardware-friendly but successfully alleviate the dying neuron issue. Besides, each series includes a number of members, which can be freely selected through programming to best fit a given application. Experimental results indicate that the proposed new activation functions achieve comparable or even better model accuracy as compared to GELU. Moreover, the highly efficient and flexible VLSI implementations support 16 different Q-formats to maximize the output precision under various input scales. Compared with approximation-based implementation strategies, the proposed activation function designs and the corresponding LUT-free hardware implementations do achieve a significant improvement in speed, area, and power." @default.
- W4383501712 created "2023-07-08" @default.
- W4383501712 creator A5036029066 @default.
- W4383501712 creator A5048096590 @default.
- W4383501712 creator A5091713952 @default.
- W4383501712 date "2023-06-11" @default.
- W4383501712 modified "2023-09-27" @default.
- W4383501712 title "Hardware-Friendly Activation Function Designs and Its Efficient VLSI Implementations for Transformer-Based Applications" @default.
- W4383501712 cites W2000956074 @default.
- W4383501712 cites W2071654592 @default.
- W4383501712 cites W2167425979 @default.
- W4383501712 cites W2896478499 @default.
- W4383501712 cites W2943362227 @default.
- W4383501712 cites W2965942869 @default.
- W4383501712 cites W3032892975 @default.
- W4383501712 cites W3054316493 @default.
- W4383501712 cites W3138516171 @default.
- W4383501712 cites W3160282718 @default.
- W4383501712 cites W3160664745 @default.
- W4383501712 cites W3164250128 @default.
- W4383501712 cites W3164612836 @default.
- W4383501712 cites W3217128056 @default.
- W4383501712 cites W4206677985 @default.
- W4383501712 cites W4210722876 @default.
- W4383501712 doi "https://doi.org/10.1109/aicas57966.2023.10168591" @default.
- W4383501712 hasPublicationYear "2023" @default.
- W4383501712 type Work @default.
- W4383501712 citedByCount "0" @default.
- W4383501712 crossrefType "proceedings-article" @default.
- W4383501712 hasAuthorship W4383501712A5036029066 @default.
- W4383501712 hasAuthorship W4383501712A5048096590 @default.
- W4383501712 hasAuthorship W4383501712A5091713952 @default.
- W4383501712 hasConcept C113775141 @default.
- W4383501712 hasConcept C118524514 @default.
- W4383501712 hasConcept C119599485 @default.
- W4383501712 hasConcept C127413603 @default.
- W4383501712 hasConcept C134835016 @default.
- W4383501712 hasConcept C14580979 @default.
- W4383501712 hasConcept C149635348 @default.
- W4383501712 hasConcept C154945302 @default.
- W4383501712 hasConcept C165801399 @default.
- W4383501712 hasConcept C173608175 @default.
- W4383501712 hasConcept C199360897 @default.
- W4383501712 hasConcept C26713055 @default.
- W4383501712 hasConcept C38365724 @default.
- W4383501712 hasConcept C41008148 @default.
- W4383501712 hasConcept C50644808 @default.
- W4383501712 hasConcept C66322947 @default.
- W4383501712 hasConceptScore W4383501712C113775141 @default.
- W4383501712 hasConceptScore W4383501712C118524514 @default.
- W4383501712 hasConceptScore W4383501712C119599485 @default.
- W4383501712 hasConceptScore W4383501712C127413603 @default.
- W4383501712 hasConceptScore W4383501712C134835016 @default.
- W4383501712 hasConceptScore W4383501712C14580979 @default.
- W4383501712 hasConceptScore W4383501712C149635348 @default.
- W4383501712 hasConceptScore W4383501712C154945302 @default.
- W4383501712 hasConceptScore W4383501712C165801399 @default.
- W4383501712 hasConceptScore W4383501712C173608175 @default.
- W4383501712 hasConceptScore W4383501712C199360897 @default.
- W4383501712 hasConceptScore W4383501712C26713055 @default.
- W4383501712 hasConceptScore W4383501712C38365724 @default.
- W4383501712 hasConceptScore W4383501712C41008148 @default.
- W4383501712 hasConceptScore W4383501712C50644808 @default.
- W4383501712 hasConceptScore W4383501712C66322947 @default.
- W4383501712 hasFunder F4320331164 @default.
- W4383501712 hasLocation W43835017121 @default.
- W4383501712 hasOpenAccess W4383501712 @default.
- W4383501712 hasPrimaryLocation W43835017121 @default.
- W4383501712 hasRelatedWork W1617740971 @default.
- W4383501712 hasRelatedWork W1996849963 @default.
- W4383501712 hasRelatedWork W2066555863 @default.
- W4383501712 hasRelatedWork W2101501703 @default.
- W4383501712 hasRelatedWork W2120840712 @default.
- W4383501712 hasRelatedWork W2134271968 @default.
- W4383501712 hasRelatedWork W2793542907 @default.
- W4383501712 hasRelatedWork W310235667 @default.
- W4383501712 hasRelatedWork W3186509556 @default.
- W4383501712 hasRelatedWork W79134960 @default.
- W4383501712 isParatext "false" @default.
- W4383501712 isRetracted "false" @default.
- W4383501712 workType "article" @default.