Matches in SemOpenAlex for { <https://semopenalex.org/work/W3080441504> ?p ?o ?g. }
Showing items 1 to 97 of
97
with 100 items per page.
- W3080441504 abstract "Deep neural networks achieve state-of-the-art performance in a variety of tasks, however this performance is closely tied to model size. Sparsity is one approach to limiting model size. Modern techniques for inducing sparsity in neural networks are (1) network pruning, a procedure involving iteratively training a model initialized with a previous run's weights and hard thresholding, (2) training in one-stage with a sparsity inducing penalty (usually based on the Lasso), and (3) training a binary mask jointly with the weights of the network. In this work, we study different sparsity inducing penalties from the perspective of Bayesian hierarchical models with the goal of designing penalties which perform well without retraining subnetworks in isolation. With this motivation, we present a novel penalty called Hierarchical Adaptive Lasso (HALO) which learns to adaptively sparsify weights of a given network via trainable parameters without learning a mask. When used to train over-parametrized networks, our penalty yields small subnetworks with high accuracy (winning tickets) even when the subnetworks are not trained in isolation. Empirically, on the CIFAR-100 dataset, we find that HALO is able to learn highly sparse network (only $5%$ of the parameters) with approximately a $2%$ and $4%$ gain in performance over state-of-the-art magnitude pruning methods at the same level of sparsity." @default.
- W3080441504 created "2020-09-01" @default.
- W3080441504 creator A5042419038 @default.
- W3080441504 creator A5059839283 @default.
- W3080441504 creator A5084409300 @default.
- W3080441504 date "2020-08-24" @default.
- W3080441504 modified "2023-09-27" @default.
- W3080441504 title "Hierarchical Adaptive Lasso: Learning Sparse Neural Networks with Shrinkage via Single Stage Training." @default.
- W3080441504 cites W1529151133 @default.
- W3080441504 cites W1975172027 @default.
- W3080441504 cites W1999974018 @default.
- W3080441504 cites W2004810039 @default.
- W3080441504 cites W2020925091 @default.
- W3080441504 cites W2028069051 @default.
- W3080441504 cites W2062532221 @default.
- W3080441504 cites W2074682976 @default.
- W3080441504 cites W2096764219 @default.
- W3080441504 cites W2114766824 @default.
- W3080441504 cites W2120160881 @default.
- W3080441504 cites W2127300249 @default.
- W3080441504 cites W2135046866 @default.
- W3080441504 cites W2147656689 @default.
- W3080441504 cites W2150940164 @default.
- W3080441504 cites W2156150815 @default.
- W3080441504 cites W2160815625 @default.
- W3080441504 cites W2223242284 @default.
- W3080441504 cites W2786237224 @default.
- W3080441504 cites W2905810301 @default.
- W3080441504 cites W2943283198 @default.
- W3080441504 cites W2949701357 @default.
- W3080441504 cites W2962978766 @default.
- W3080441504 cites W2963341956 @default.
- W3080441504 cites W2963363373 @default.
- W3080441504 cites W2963382930 @default.
- W3080441504 cites W2964152344 @default.
- W3080441504 cites W2964299589 @default.
- W3080441504 cites W2982552840 @default.
- W3080441504 cites W3024084317 @default.
- W3080441504 cites W3101685967 @default.
- W3080441504 cites W3104393726 @default.
- W3080441504 hasPublicationYear "2020" @default.
- W3080441504 type Work @default.
- W3080441504 sameAs 3080441504 @default.
- W3080441504 citedByCount "0" @default.
- W3080441504 crossrefType "posted-content" @default.
- W3080441504 hasAuthorship W3080441504A5042419038 @default.
- W3080441504 hasAuthorship W3080441504A5059839283 @default.
- W3080441504 hasAuthorship W3080441504A5084409300 @default.
- W3080441504 hasConcept C108010975 @default.
- W3080441504 hasConcept C119857082 @default.
- W3080441504 hasConcept C136764020 @default.
- W3080441504 hasConcept C154945302 @default.
- W3080441504 hasConcept C22019652 @default.
- W3080441504 hasConcept C2776145597 @default.
- W3080441504 hasConcept C37616216 @default.
- W3080441504 hasConcept C41008148 @default.
- W3080441504 hasConcept C50644808 @default.
- W3080441504 hasConcept C6557445 @default.
- W3080441504 hasConcept C86803240 @default.
- W3080441504 hasConceptScore W3080441504C108010975 @default.
- W3080441504 hasConceptScore W3080441504C119857082 @default.
- W3080441504 hasConceptScore W3080441504C136764020 @default.
- W3080441504 hasConceptScore W3080441504C154945302 @default.
- W3080441504 hasConceptScore W3080441504C22019652 @default.
- W3080441504 hasConceptScore W3080441504C2776145597 @default.
- W3080441504 hasConceptScore W3080441504C37616216 @default.
- W3080441504 hasConceptScore W3080441504C41008148 @default.
- W3080441504 hasConceptScore W3080441504C50644808 @default.
- W3080441504 hasConceptScore W3080441504C6557445 @default.
- W3080441504 hasConceptScore W3080441504C86803240 @default.
- W3080441504 hasLocation W30804415041 @default.
- W3080441504 hasOpenAccess W3080441504 @default.
- W3080441504 hasPrimaryLocation W30804415041 @default.
- W3080441504 hasRelatedWork W2276892413 @default.
- W3080441504 hasRelatedWork W2731589127 @default.
- W3080441504 hasRelatedWork W2785668315 @default.
- W3080441504 hasRelatedWork W2903956504 @default.
- W3080441504 hasRelatedWork W2911419447 @default.
- W3080441504 hasRelatedWork W2911970084 @default.
- W3080441504 hasRelatedWork W3005908409 @default.
- W3080441504 hasRelatedWork W3024929536 @default.
- W3080441504 hasRelatedWork W3025590998 @default.
- W3080441504 hasRelatedWork W3081131905 @default.
- W3080441504 hasRelatedWork W3107352446 @default.
- W3080441504 hasRelatedWork W3117969921 @default.
- W3080441504 hasRelatedWork W3121699489 @default.
- W3080441504 hasRelatedWork W3123546926 @default.
- W3080441504 hasRelatedWork W3128194313 @default.
- W3080441504 hasRelatedWork W3155573568 @default.
- W3080441504 hasRelatedWork W3158839075 @default.
- W3080441504 hasRelatedWork W3200914073 @default.
- W3080441504 hasRelatedWork W3205684019 @default.
- W3080441504 hasRelatedWork W3119992913 @default.
- W3080441504 isParatext "false" @default.
- W3080441504 isRetracted "false" @default.
- W3080441504 magId "3080441504" @default.
- W3080441504 workType "article" @default.