Matches in SemOpenAlex for { <https://semopenalex.org/work/W4323905478> ?p ?o ?g. }
Showing items 1 to 94 of
94
with 100 items per page.
- W4323905478 endingPage "465" @default.
- W4323905478 startingPage "389" @default.
- W4323905478 abstract "We study the effect of normalization on the layers of deep neural networks of feed-forward type. A given layer $ i $ with $ N_{i} $ hidden units is allowed to be normalized by $ 1/N_{i}^{gamma_{i}} $ with $ gamma_{i}in[1/2,1] $ and we study the effect of the choice of the $ gamma_{i} $ on the statistical behavior of the neural network's output (such as variance) as well as on the test accuracy on the MNIST data set. We find that in terms of variance of the neural network's output and test accuracy the best choice is to choose the $ gamma_{i} $'s to be equal to one, which is the mean-field scaling. We also find that this is particularly true for the outer layer, in that the neural network's behavior is more sensitive in the scaling of the outer layer as opposed to the scaling of the inner layers. The mechanism for the mathematical analysis is an asymptotic expansion for the neural network's output. An important practical consequence of the analysis is that it provides a systematic and mathematically informed way to choose the learning rate hyperparameters. Such a choice guarantees that the neural network behaves in a statistically robust way as the $ N_i $ grow to infinity." @default.
- W4323905478 created "2023-03-12" @default.
- W4323905478 creator A5001205723 @default.
- W4323905478 creator A5073501417 @default.
- W4323905478 date "2023-01-01" @default.
- W4323905478 modified "2023-09-30" @default.
- W4323905478 title "Normalization effects on deep neural networks" @default.
- W4323905478 cites W1019830208 @default.
- W4323905478 cites W1988115241 @default.
- W4323905478 cites W2016833839 @default.
- W4323905478 cites W2086267868 @default.
- W4323905478 cites W2112796928 @default.
- W4323905478 cites W2137983211 @default.
- W4323905478 cites W2145287260 @default.
- W4323905478 cites W2145708596 @default.
- W4323905478 cites W2261689926 @default.
- W4323905478 cites W2345737627 @default.
- W4323905478 cites W2530876040 @default.
- W4323905478 cites W2534240011 @default.
- W4323905478 cites W2581082771 @default.
- W4323905478 cites W2736506089 @default.
- W4323905478 cites W2749028154 @default.
- W4323905478 cites W2907047316 @default.
- W4323905478 cites W2919115771 @default.
- W4323905478 cites W2963095610 @default.
- W4323905478 cites W2963751193 @default.
- W4323905478 cites W2963791871 @default.
- W4323905478 cites W3010825589 @default.
- W4323905478 cites W3155817928 @default.
- W4323905478 cites W3171655036 @default.
- W4323905478 cites W3211566408 @default.
- W4323905478 cites W4213329537 @default.
- W4323905478 cites W4238558850 @default.
- W4323905478 doi "https://doi.org/10.3934/fods.2023004" @default.
- W4323905478 hasPublicationYear "2023" @default.
- W4323905478 type Work @default.
- W4323905478 citedByCount "0" @default.
- W4323905478 crossrefType "journal-article" @default.
- W4323905478 hasAuthorship W4323905478A5001205723 @default.
- W4323905478 hasAuthorship W4323905478A5073501417 @default.
- W4323905478 hasBestOaLocation W43239054781 @default.
- W4323905478 hasConcept C11413529 @default.
- W4323905478 hasConcept C136886441 @default.
- W4323905478 hasConcept C144024400 @default.
- W4323905478 hasConcept C154945302 @default.
- W4323905478 hasConcept C169903167 @default.
- W4323905478 hasConcept C178790620 @default.
- W4323905478 hasConcept C185592680 @default.
- W4323905478 hasConcept C190502265 @default.
- W4323905478 hasConcept C19165224 @default.
- W4323905478 hasConcept C2524010 @default.
- W4323905478 hasConcept C2779227376 @default.
- W4323905478 hasConcept C33923547 @default.
- W4323905478 hasConcept C41008148 @default.
- W4323905478 hasConcept C50644808 @default.
- W4323905478 hasConcept C8642999 @default.
- W4323905478 hasConcept C99844830 @default.
- W4323905478 hasConceptScore W4323905478C11413529 @default.
- W4323905478 hasConceptScore W4323905478C136886441 @default.
- W4323905478 hasConceptScore W4323905478C144024400 @default.
- W4323905478 hasConceptScore W4323905478C154945302 @default.
- W4323905478 hasConceptScore W4323905478C169903167 @default.
- W4323905478 hasConceptScore W4323905478C178790620 @default.
- W4323905478 hasConceptScore W4323905478C185592680 @default.
- W4323905478 hasConceptScore W4323905478C190502265 @default.
- W4323905478 hasConceptScore W4323905478C19165224 @default.
- W4323905478 hasConceptScore W4323905478C2524010 @default.
- W4323905478 hasConceptScore W4323905478C2779227376 @default.
- W4323905478 hasConceptScore W4323905478C33923547 @default.
- W4323905478 hasConceptScore W4323905478C41008148 @default.
- W4323905478 hasConceptScore W4323905478C50644808 @default.
- W4323905478 hasConceptScore W4323905478C8642999 @default.
- W4323905478 hasConceptScore W4323905478C99844830 @default.
- W4323905478 hasIssue "3" @default.
- W4323905478 hasLocation W43239054781 @default.
- W4323905478 hasLocation W43239054782 @default.
- W4323905478 hasOpenAccess W4323905478 @default.
- W4323905478 hasPrimaryLocation W43239054781 @default.
- W4323905478 hasRelatedWork W3010968977 @default.
- W4323905478 hasRelatedWork W3013823630 @default.
- W4323905478 hasRelatedWork W3141609294 @default.
- W4323905478 hasRelatedWork W3198754422 @default.
- W4323905478 hasRelatedWork W3208327626 @default.
- W4323905478 hasRelatedWork W3214869653 @default.
- W4323905478 hasRelatedWork W3216553692 @default.
- W4323905478 hasRelatedWork W4287827292 @default.
- W4323905478 hasRelatedWork W4294789471 @default.
- W4323905478 hasRelatedWork W4323905478 @default.
- W4323905478 hasVolume "5" @default.
- W4323905478 isParatext "false" @default.
- W4323905478 isRetracted "false" @default.
- W4323905478 workType "article" @default.