FRINK Linked Data Fragments server

Matches in SemOpenAlex for { <https://semopenalex.org/work/W3132669550> ?p ?o ?g. }

• W3132669550 abstract "Recently, researchers proposed pruning deep neural network weights (DNNs) using an $N:M$ fine-grained block sparsity mask. In this mask, for each block of $M$ weights, we have at least $N$ zeros. In contrast to unstructured sparsity, $N:M$ fine-grained block sparsity allows acceleration in actual modern hardware. So far, this was used for DNN acceleration at the inference phase. First, we suggest a method to convert a pretrained model with unstructured sparsity to a $N:M$ fine-grained block sparsity model, with little to no training. Then, to also allow such acceleration in the training phase, we suggest a novel transposable-fine-grained sparsity mask where the same mask can be used for both forward and backward passes. Our transposable mask ensures that both the weight matrix and its transpose follow the same sparsity pattern; thus the matrix multiplication required for passing the error backward can also be accelerated. We discuss the transposable constraint and devise a new measure for mask constraints, called mask-diversity (MD), which correlates with their expected accuracy. Then, we formulate the problem of finding the optimal transposable mask as a minimum-cost-flow problem and suggest a fast linear approximation that can be used when the masks dynamically change while training. Our experiments suggest 2x speed-up with no accuracy degradation over vision and language models. A reference implementation can be found at this https URL." @default.
• W3132669550 created "2021-03-01" @default.
• W3132669550 creator A5017964819 @default.
• W3132669550 creator A5021488681 @default.
• W3132669550 creator A5025123281 @default.
• W3132669550 creator A5046332422 @default.
• W3132669550 creator A5087518651 @default.
• W3132669550 creator A5091675091 @default.
• W3132669550 date "2021-02-16" @default.
• W3132669550 modified "2023-09-27" @default.
• W3132669550 title "Accelerated Sparse Neural Training: A Provable and Efficient Method to Find N:M Transposable Masks" @default.
• W3132669550 cites W1821462560 @default.
• W3132669550 cites W2013825190 @default.
• W3132669550 cites W2025713906 @default.
• W3132669550 cites W2097533491 @default.
• W3132669550 cites W2099001231 @default.
• W3132669550 cites W2134273960 @default.
• W3132669550 cites W2194775991 @default.
• W3132669550 cites W2242818861 @default.
• W3132669550 cites W2524428287 @default.
• W3132669550 cites W2768501777 @default.
• W3132669550 cites W2769802733 @default.
• W3132669550 cites W2896718327 @default.
• W3132669550 cites W2907886210 @default.
• W3132669550 cites W2912168260 @default.
• W3132669550 cites W2948695999 @default.
• W3132669550 cites W2951569836 @default.
• W3132669550 cites W2956434358 @default.
• W3132669550 cites W2962965870 @default.
• W3132669550 cites W2963000224 @default.
• W3132669550 cites W2963247446 @default.
• W3132669550 cites W2963674932 @default.
• W3132669550 cites W2963709416 @default.
• W3132669550 cites W2963813662 @default.
• W3132669550 cites W2963828549 @default.
• W3132669550 cites W2963918968 @default.
• W3132669550 cites W2964233199 @default.
• W3132669550 cites W2967733054 @default.
• W3132669550 cites W2985339713 @default.
• W3132669550 cites W2995816250 @default.
• W3132669550 cites W3030163527 @default.
• W3132669550 cites W3034733718 @default.
• W3132669550 cites W3035078287 @default.
• W3132669550 cites W3035180000 @default.
• W3132669550 cites W3035855442 @default.
• W3132669550 cites W3083448369 @default.
• W3132669550 cites W3119866685 @default.
• W3132669550 cites W3127067080 @default.
• W3132669550 cites W3130660608 @default.
• W3132669550 cites W3165282941 @default.
• W3132669550 hasPublicationYear "2021" @default.
• W3132669550 type Work @default.
• W3132669550 sameAs 3132669550 @default.
• W3132669550 citedByCount "5" @default.
• W3132669550 countsByYear W31326695502021 @default.
• W3132669550 countsByYear W31326695502022 @default.
• W3132669550 crossrefType "posted-content" @default.
• W3132669550 hasAuthorship W3132669550A5017964819 @default.
• W3132669550 hasAuthorship W3132669550A5021488681 @default.
• W3132669550 hasAuthorship W3132669550A5025123281 @default.
• W3132669550 hasAuthorship W3132669550A5046332422 @default.
• W3132669550 hasAuthorship W3132669550A5087518651 @default.
• W3132669550 hasAuthorship W3132669550A5091675091 @default.
• W3132669550 hasConcept C104317684 @default.
• W3132669550 hasConcept C106487976 @default.
• W3132669550 hasConcept C108010975 @default.
• W3132669550 hasConcept C11413529 @default.
• W3132669550 hasConcept C117896860 @default.
• W3132669550 hasConcept C121332964 @default.
• W3132669550 hasConcept C141231307 @default.
• W3132669550 hasConcept C154945302 @default.
• W3132669550 hasConcept C158693339 @default.
• W3132669550 hasConcept C159985019 @default.
• W3132669550 hasConcept C185592680 @default.
• W3132669550 hasConcept C192562407 @default.
• W3132669550 hasConcept C200106649 @default.
• W3132669550 hasConcept C2524010 @default.
• W3132669550 hasConcept C2776036281 @default.
• W3132669550 hasConcept C2776214188 @default.
• W3132669550 hasConcept C2777210771 @default.
• W3132669550 hasConcept C2984842247 @default.
• W3132669550 hasConcept C33923547 @default.
• W3132669550 hasConcept C41008148 @default.
• W3132669550 hasConcept C4918238 @default.
• W3132669550 hasConcept C50644808 @default.
• W3132669550 hasConcept C55493867 @default.
• W3132669550 hasConcept C62520636 @default.
• W3132669550 hasConcept C6557445 @default.
• W3132669550 hasConcept C74650414 @default.
• W3132669550 hasConcept C86803240 @default.
• W3132669550 hasConceptScore W3132669550C104317684 @default.
• W3132669550 hasConceptScore W3132669550C106487976 @default.
• W3132669550 hasConceptScore W3132669550C108010975 @default.
• W3132669550 hasConceptScore W3132669550C11413529 @default.
• W3132669550 hasConceptScore W3132669550C117896860 @default.
• W3132669550 hasConceptScore W3132669550C121332964 @default.
• W3132669550 hasConceptScore W3132669550C141231307 @default.
• W3132669550 hasConceptScore W3132669550C154945302 @default.
• W3132669550 hasConceptScore W3132669550C158693339 @default.
• W3132669550 hasConceptScore W3132669550C159985019 @default.

• next