Matches in SemOpenAlex for { <https://semopenalex.org/work/W4289600961> ?p ?o ?g. }
Showing items 1 to 73 of
73
with 100 items per page.
- W4289600961 abstract "It is well-known that the Hessian of deep loss landscape matters to optimization, generalization, and even robustness of deep learning. Recent works empirically discovered that the Hessian spectrum in deep learning has a two-component structure that consists of a small number of large eigenvalues and a large number of nearly-zero eigenvalues. However, the theoretical mechanism or the mathematical behind the Hessian spectrum is still largely under-explored. To the best of our knowledge, we are the first to demonstrate that the Hessian spectrums of well-trained deep neural networks exhibit simple power-law structures. Inspired by the statistical physical theories and the spectral analysis of natural proteins, we provide a maximum-entropy theoretical interpretation for explaining why the power-law structure exist and suggest a spectral parallel between protein evolution and training of deep neural networks. By conducing extensive experiments, we further use the power-law spectral framework as a useful tool to explore multiple novel behaviors of deep learning." @default.
- W4289600961 created "2022-08-03" @default.
- W4289600961 creator A5005641904 @default.
- W4289600961 creator A5013531859 @default.
- W4289600961 creator A5017058023 @default.
- W4289600961 creator A5066773635 @default.
- W4289600961 creator A5073070877 @default.
- W4289600961 date "2022-01-31" @default.
- W4289600961 modified "2023-10-16" @default.
- W4289600961 title "On the Power-Law Hessian Spectrums in Deep Learning" @default.
- W4289600961 doi "https://doi.org/10.48550/arxiv.2201.13011" @default.
- W4289600961 hasPublicationYear "2022" @default.
- W4289600961 type Work @default.
- W4289600961 citedByCount "0" @default.
- W4289600961 crossrefType "posted-content" @default.
- W4289600961 hasAuthorship W4289600961A5005641904 @default.
- W4289600961 hasAuthorship W4289600961A5013531859 @default.
- W4289600961 hasAuthorship W4289600961A5017058023 @default.
- W4289600961 hasAuthorship W4289600961A5066773635 @default.
- W4289600961 hasAuthorship W4289600961A5073070877 @default.
- W4289600961 hasBestOaLocation W42896009611 @default.
- W4289600961 hasConcept C104317684 @default.
- W4289600961 hasConcept C108583219 @default.
- W4289600961 hasConcept C119857082 @default.
- W4289600961 hasConcept C121332964 @default.
- W4289600961 hasConcept C121864883 @default.
- W4289600961 hasConcept C154945302 @default.
- W4289600961 hasConcept C158693339 @default.
- W4289600961 hasConcept C17744445 @default.
- W4289600961 hasConcept C185592680 @default.
- W4289600961 hasConcept C199539241 @default.
- W4289600961 hasConcept C203616005 @default.
- W4289600961 hasConcept C28826006 @default.
- W4289600961 hasConcept C33923547 @default.
- W4289600961 hasConcept C41008148 @default.
- W4289600961 hasConcept C50644808 @default.
- W4289600961 hasConcept C55493867 @default.
- W4289600961 hasConcept C62520636 @default.
- W4289600961 hasConcept C63479239 @default.
- W4289600961 hasConceptScore W4289600961C104317684 @default.
- W4289600961 hasConceptScore W4289600961C108583219 @default.
- W4289600961 hasConceptScore W4289600961C119857082 @default.
- W4289600961 hasConceptScore W4289600961C121332964 @default.
- W4289600961 hasConceptScore W4289600961C121864883 @default.
- W4289600961 hasConceptScore W4289600961C154945302 @default.
- W4289600961 hasConceptScore W4289600961C158693339 @default.
- W4289600961 hasConceptScore W4289600961C17744445 @default.
- W4289600961 hasConceptScore W4289600961C185592680 @default.
- W4289600961 hasConceptScore W4289600961C199539241 @default.
- W4289600961 hasConceptScore W4289600961C203616005 @default.
- W4289600961 hasConceptScore W4289600961C28826006 @default.
- W4289600961 hasConceptScore W4289600961C33923547 @default.
- W4289600961 hasConceptScore W4289600961C41008148 @default.
- W4289600961 hasConceptScore W4289600961C50644808 @default.
- W4289600961 hasConceptScore W4289600961C55493867 @default.
- W4289600961 hasConceptScore W4289600961C62520636 @default.
- W4289600961 hasConceptScore W4289600961C63479239 @default.
- W4289600961 hasLocation W42896009611 @default.
- W4289600961 hasOpenAccess W4289600961 @default.
- W4289600961 hasPrimaryLocation W42896009611 @default.
- W4289600961 hasRelatedWork W11300528 @default.
- W4289600961 hasRelatedWork W11339170 @default.
- W4289600961 hasRelatedWork W180746 @default.
- W4289600961 hasRelatedWork W3647669 @default.
- W4289600961 hasRelatedWork W3657516 @default.
- W4289600961 hasRelatedWork W5477720 @default.
- W4289600961 hasRelatedWork W5731987 @default.
- W4289600961 hasRelatedWork W9190101 @default.
- W4289600961 hasRelatedWork W9948473 @default.
- W4289600961 hasRelatedWork W10152789 @default.
- W4289600961 isParatext "false" @default.
- W4289600961 isRetracted "false" @default.
- W4289600961 workType "article" @default.