Matches in SemOpenAlex for { <https://semopenalex.org/work/W4384615935> ?p ?o ?g. }
Showing items 1 to 83 of
83
with 100 items per page.
- W4384615935 abstract "The sparsity of Deep Neural Networks is well investigated to maximize the performance and reduce the size of overparameterized networks as possible. Existing methods focus on pruning parameters in the training process by using thresholds and metrics. Meanwhile, feature similarity between different layers has not been discussed sufficiently before, which could be rigorously proved to be highly correlated to the network sparsity in this paper. Inspired by interlayer feature similarity in overparameterized models, we investigate the intrinsic link between network sparsity and interlayer feature similarity. Specifically, we prove that reducing interlayer feature similarity based on Centered Kernel Alignment (CKA) improves the sparsity of the network by using information bottleneck theory. Applying such theory, we propose a plug-and-play CKA-based Sparsity Regularization for sparse network training, dubbed CKA-SR, which utilizes CKA to reduce feature similarity between layers and increase network sparsity. In other words, layers of our sparse network tend to have their own identity compared to each other. Experimentally, we plug the proposed CKA-SR into the training process of sparse network training methods and find that CKA-SR consistently improves the performance of several State-Of-The-Art sparse training methods, especially at extremely high sparsity. Code is included in the supplementary materials." @default.
- W4384615935 created "2023-07-18" @default.
- W4384615935 creator A5013848428 @default.
- W4384615935 creator A5046214153 @default.
- W4384615935 creator A5046673071 @default.
- W4384615935 creator A5054226277 @default.
- W4384615935 creator A5070755370 @default.
- W4384615935 creator A5087041063 @default.
- W4384615935 date "2023-07-14" @default.
- W4384615935 modified "2023-09-23" @default.
- W4384615935 title "Learning Sparse Neural Networks with Identity Layers" @default.
- W4384615935 doi "https://doi.org/10.48550/arxiv.2307.07389" @default.
- W4384615935 hasPublicationYear "2023" @default.
- W4384615935 type Work @default.
- W4384615935 citedByCount "0" @default.
- W4384615935 crossrefType "posted-content" @default.
- W4384615935 hasAuthorship W4384615935A5013848428 @default.
- W4384615935 hasAuthorship W4384615935A5046214153 @default.
- W4384615935 hasAuthorship W4384615935A5046673071 @default.
- W4384615935 hasAuthorship W4384615935A5054226277 @default.
- W4384615935 hasAuthorship W4384615935A5070755370 @default.
- W4384615935 hasAuthorship W4384615935A5087041063 @default.
- W4384615935 hasBestOaLocation W43846159351 @default.
- W4384615935 hasConcept C103278499 @default.
- W4384615935 hasConcept C108010975 @default.
- W4384615935 hasConcept C11413529 @default.
- W4384615935 hasConcept C114614502 @default.
- W4384615935 hasConcept C115961682 @default.
- W4384615935 hasConcept C120665830 @default.
- W4384615935 hasConcept C121332964 @default.
- W4384615935 hasConcept C138885662 @default.
- W4384615935 hasConcept C149635348 @default.
- W4384615935 hasConcept C153180895 @default.
- W4384615935 hasConcept C154945302 @default.
- W4384615935 hasConcept C192209626 @default.
- W4384615935 hasConcept C2776135515 @default.
- W4384615935 hasConcept C2776401178 @default.
- W4384615935 hasConcept C2780513914 @default.
- W4384615935 hasConcept C33923547 @default.
- W4384615935 hasConcept C41008148 @default.
- W4384615935 hasConcept C41895202 @default.
- W4384615935 hasConcept C50644808 @default.
- W4384615935 hasConcept C6557445 @default.
- W4384615935 hasConcept C74193536 @default.
- W4384615935 hasConcept C86803240 @default.
- W4384615935 hasConceptScore W4384615935C103278499 @default.
- W4384615935 hasConceptScore W4384615935C108010975 @default.
- W4384615935 hasConceptScore W4384615935C11413529 @default.
- W4384615935 hasConceptScore W4384615935C114614502 @default.
- W4384615935 hasConceptScore W4384615935C115961682 @default.
- W4384615935 hasConceptScore W4384615935C120665830 @default.
- W4384615935 hasConceptScore W4384615935C121332964 @default.
- W4384615935 hasConceptScore W4384615935C138885662 @default.
- W4384615935 hasConceptScore W4384615935C149635348 @default.
- W4384615935 hasConceptScore W4384615935C153180895 @default.
- W4384615935 hasConceptScore W4384615935C154945302 @default.
- W4384615935 hasConceptScore W4384615935C192209626 @default.
- W4384615935 hasConceptScore W4384615935C2776135515 @default.
- W4384615935 hasConceptScore W4384615935C2776401178 @default.
- W4384615935 hasConceptScore W4384615935C2780513914 @default.
- W4384615935 hasConceptScore W4384615935C33923547 @default.
- W4384615935 hasConceptScore W4384615935C41008148 @default.
- W4384615935 hasConceptScore W4384615935C41895202 @default.
- W4384615935 hasConceptScore W4384615935C50644808 @default.
- W4384615935 hasConceptScore W4384615935C6557445 @default.
- W4384615935 hasConceptScore W4384615935C74193536 @default.
- W4384615935 hasConceptScore W4384615935C86803240 @default.
- W4384615935 hasLocation W43846159351 @default.
- W4384615935 hasOpenAccess W4384615935 @default.
- W4384615935 hasPrimaryLocation W43846159351 @default.
- W4384615935 hasRelatedWork W2015538044 @default.
- W4384615935 hasRelatedWork W2016461833 @default.
- W4384615935 hasRelatedWork W2052253960 @default.
- W4384615935 hasRelatedWork W2110459882 @default.
- W4384615935 hasRelatedWork W2118043379 @default.
- W4384615935 hasRelatedWork W2147802381 @default.
- W4384615935 hasRelatedWork W2151022383 @default.
- W4384615935 hasRelatedWork W2382607599 @default.
- W4384615935 hasRelatedWork W2546942002 @default.
- W4384615935 hasRelatedWork W2760085659 @default.
- W4384615935 isParatext "false" @default.
- W4384615935 isRetracted "false" @default.
- W4384615935 workType "article" @default.