Matches in SemOpenAlex for { <https://semopenalex.org/work/W4290802700> ?p ?o ?g. }
Showing items 1 to 63 of
63
with 100 items per page.
- W4290802700 abstract "Neural networks are universal function approximators which are known to generalize well despite being dramatically overparameterized. We study this phenomenon from the point of view of the spectral bias of neural networks. Our contributions are two-fold. First, we provide a theoretical explanation for the spectral bias of ReLU neural networks by leveraging connections with the theory of finite element methods. Second, based upon this theory we predict that switching the activation function to a piecewise linear B-spline, namely the Hat function, will remove this spectral bias, which we verify empirically in a variety of settings. Our empirical studies also show that neural networks with the Hat activation function are trained significantly faster using stochastic gradient descent and ADAM. Combined with previous work showing that the Hat activation function also improves generalization accuracy on image classification tasks, this indicates that using the Hat activation provides significant advantages over the ReLU on certain problems." @default.
- W4290802700 created "2022-08-12" @default.
- W4290802700 creator A5014166483 @default.
- W4290802700 creator A5014378909 @default.
- W4290802700 creator A5071400648 @default.
- W4290802700 creator A5072176149 @default.
- W4290802700 date "2022-08-09" @default.
- W4290802700 modified "2023-10-14" @default.
- W4290802700 title "On the Activation Function Dependence of the Spectral Bias of Neural Networks" @default.
- W4290802700 doi "https://doi.org/10.48550/arxiv.2208.04924" @default.
- W4290802700 hasPublicationYear "2022" @default.
- W4290802700 type Work @default.
- W4290802700 citedByCount "0" @default.
- W4290802700 crossrefType "posted-content" @default.
- W4290802700 hasAuthorship W4290802700A5014166483 @default.
- W4290802700 hasAuthorship W4290802700A5014378909 @default.
- W4290802700 hasAuthorship W4290802700A5071400648 @default.
- W4290802700 hasAuthorship W4290802700A5072176149 @default.
- W4290802700 hasBestOaLocation W42908027001 @default.
- W4290802700 hasConcept C11413529 @default.
- W4290802700 hasConcept C134306372 @default.
- W4290802700 hasConcept C14036430 @default.
- W4290802700 hasConcept C153258448 @default.
- W4290802700 hasConcept C154945302 @default.
- W4290802700 hasConcept C164660894 @default.
- W4290802700 hasConcept C17095337 @default.
- W4290802700 hasConcept C177148314 @default.
- W4290802700 hasConcept C33923547 @default.
- W4290802700 hasConcept C38365724 @default.
- W4290802700 hasConcept C41008148 @default.
- W4290802700 hasConcept C50644808 @default.
- W4290802700 hasConcept C78458016 @default.
- W4290802700 hasConcept C86803240 @default.
- W4290802700 hasConceptScore W4290802700C11413529 @default.
- W4290802700 hasConceptScore W4290802700C134306372 @default.
- W4290802700 hasConceptScore W4290802700C14036430 @default.
- W4290802700 hasConceptScore W4290802700C153258448 @default.
- W4290802700 hasConceptScore W4290802700C154945302 @default.
- W4290802700 hasConceptScore W4290802700C164660894 @default.
- W4290802700 hasConceptScore W4290802700C17095337 @default.
- W4290802700 hasConceptScore W4290802700C177148314 @default.
- W4290802700 hasConceptScore W4290802700C33923547 @default.
- W4290802700 hasConceptScore W4290802700C38365724 @default.
- W4290802700 hasConceptScore W4290802700C41008148 @default.
- W4290802700 hasConceptScore W4290802700C50644808 @default.
- W4290802700 hasConceptScore W4290802700C78458016 @default.
- W4290802700 hasConceptScore W4290802700C86803240 @default.
- W4290802700 hasLocation W42908027001 @default.
- W4290802700 hasOpenAccess W4290802700 @default.
- W4290802700 hasPrimaryLocation W42908027001 @default.
- W4290802700 hasRelatedWork W1517416806 @default.
- W4290802700 hasRelatedWork W2042196920 @default.
- W4290802700 hasRelatedWork W2893539081 @default.
- W4290802700 hasRelatedWork W2963920515 @default.
- W4290802700 hasRelatedWork W3125537303 @default.
- W4290802700 hasRelatedWork W3128307059 @default.
- W4290802700 hasRelatedWork W3131933975 @default.
- W4290802700 hasRelatedWork W3176948818 @default.
- W4290802700 hasRelatedWork W4205612728 @default.
- W4290802700 hasRelatedWork W4287628154 @default.
- W4290802700 isParatext "false" @default.
- W4290802700 isRetracted "false" @default.
- W4290802700 workType "article" @default.