Matches in SemOpenAlex for { <https://semopenalex.org/work/W4309232451> ?p ?o ?g. }
Showing items 1 to 91 of
91
with 100 items per page.
- W4309232451 endingPage "109" @default.
- W4309232451 startingPage "85" @default.
- W4309232451 abstract "Abstract Free Probability Theory (FPT) provides rich knowledge for handling mathematical difficulties caused by random matrices appearing in research related to deep neural networks (DNNs), such as the dynamical isometry, Fisher information matrix, and training dynamics. FPT suits these researches because the DNN’s parameter-Jacobian and input-Jacobian are polynomials of layerwise Jacobians. However, the critical assumption of asymptotic freeness of the layerwise Jacobian has not been proven mathematically so far. The asymptotic freeness assumption plays a fundamental role when propagating spectral distributions through the layers. Haar distributed orthogonal matrices are essential for achieving dynamical isometry. In this work, we prove asymptotic freeness of layerwise Jacobians of multilayer perceptron (MLP) in this case. A key to the proof is an invariance of the MLP. Considering the orthogonal matrices that fix the hidden units in each layer, we replace each layer’s parameter matrix with itself multiplied by the orthogonal matrix, and then the MLP does not change. Furthermore, if the original weights are Haar orthogonal, the Jacobian is also unchanged by this replacement. Lastly, we can replace each weight with a Haar orthogonal random matrix independent of the Jacobian of the activation function using this key fact." @default.
- W4309232451 created "2022-11-24" @default.
- W4309232451 creator A5038591719 @default.
- W4309232451 creator A5061058801 @default.
- W4309232451 date "2022-11-17" @default.
- W4309232451 modified "2023-09-29" @default.
- W4309232451 title "Asymptotic Freeness of Layerwise Jacobians Caused by Invariance of Multilayer Perceptron: The Haar Orthogonal Case" @default.
- W4309232451 cites W155856821 @default.
- W4309232451 cites W1884490694 @default.
- W4309232451 cites W1992774725 @default.
- W4309232451 cites W2021115151 @default.
- W4309232451 cites W2051302879 @default.
- W4309232451 cites W2105310469 @default.
- W4309232451 cites W2317486283 @default.
- W4309232451 cites W2485135680 @default.
- W4309232451 cites W2919115771 @default.
- W4309232451 cites W4232753567 @default.
- W4309232451 doi "https://doi.org/10.1007/s00220-022-04441-7" @default.
- W4309232451 hasPublicationYear "2022" @default.
- W4309232451 type Work @default.
- W4309232451 citedByCount "0" @default.
- W4309232451 crossrefType "journal-article" @default.
- W4309232451 hasAuthorship W4309232451A5038591719 @default.
- W4309232451 hasAuthorship W4309232451A5061058801 @default.
- W4309232451 hasBestOaLocation W43092324511 @default.
- W4309232451 hasConcept C10628310 @default.
- W4309232451 hasConcept C106487976 @default.
- W4309232451 hasConcept C121332964 @default.
- W4309232451 hasConcept C134306372 @default.
- W4309232451 hasConcept C135925592 @default.
- W4309232451 hasConcept C154945302 @default.
- W4309232451 hasConcept C158693339 @default.
- W4309232451 hasConcept C159985019 @default.
- W4309232451 hasConcept C169756996 @default.
- W4309232451 hasConcept C187064257 @default.
- W4309232451 hasConcept C192562407 @default.
- W4309232451 hasConcept C200331156 @default.
- W4309232451 hasConcept C202444582 @default.
- W4309232451 hasConcept C28826006 @default.
- W4309232451 hasConcept C33923547 @default.
- W4309232451 hasConcept C41008148 @default.
- W4309232451 hasConcept C44292817 @default.
- W4309232451 hasConcept C50644808 @default.
- W4309232451 hasConcept C60908668 @default.
- W4309232451 hasConcept C62520636 @default.
- W4309232451 hasConcept C64812099 @default.
- W4309232451 hasConceptScore W4309232451C10628310 @default.
- W4309232451 hasConceptScore W4309232451C106487976 @default.
- W4309232451 hasConceptScore W4309232451C121332964 @default.
- W4309232451 hasConceptScore W4309232451C134306372 @default.
- W4309232451 hasConceptScore W4309232451C135925592 @default.
- W4309232451 hasConceptScore W4309232451C154945302 @default.
- W4309232451 hasConceptScore W4309232451C158693339 @default.
- W4309232451 hasConceptScore W4309232451C159985019 @default.
- W4309232451 hasConceptScore W4309232451C169756996 @default.
- W4309232451 hasConceptScore W4309232451C187064257 @default.
- W4309232451 hasConceptScore W4309232451C192562407 @default.
- W4309232451 hasConceptScore W4309232451C200331156 @default.
- W4309232451 hasConceptScore W4309232451C202444582 @default.
- W4309232451 hasConceptScore W4309232451C28826006 @default.
- W4309232451 hasConceptScore W4309232451C33923547 @default.
- W4309232451 hasConceptScore W4309232451C41008148 @default.
- W4309232451 hasConceptScore W4309232451C44292817 @default.
- W4309232451 hasConceptScore W4309232451C50644808 @default.
- W4309232451 hasConceptScore W4309232451C60908668 @default.
- W4309232451 hasConceptScore W4309232451C62520636 @default.
- W4309232451 hasConceptScore W4309232451C64812099 @default.
- W4309232451 hasFunder F4320334764 @default.
- W4309232451 hasFunder F4320334789 @default.
- W4309232451 hasIssue "1" @default.
- W4309232451 hasLocation W43092324511 @default.
- W4309232451 hasLocation W43092324512 @default.
- W4309232451 hasLocation W43092324513 @default.
- W4309232451 hasOpenAccess W4309232451 @default.
- W4309232451 hasPrimaryLocation W43092324511 @default.
- W4309232451 hasRelatedWork W1967268621 @default.
- W4309232451 hasRelatedWork W2050475615 @default.
- W4309232451 hasRelatedWork W2356908423 @default.
- W4309232451 hasRelatedWork W2923568641 @default.
- W4309232451 hasRelatedWork W2937295267 @default.
- W4309232451 hasRelatedWork W3118451326 @default.
- W4309232451 hasRelatedWork W3138671610 @default.
- W4309232451 hasRelatedWork W4226115390 @default.
- W4309232451 hasRelatedWork W4240917246 @default.
- W4309232451 hasRelatedWork W4309232451 @default.
- W4309232451 hasVolume "397" @default.
- W4309232451 isParatext "false" @default.
- W4309232451 isRetracted "false" @default.
- W4309232451 workType "article" @default.