FRINK Linked Data Fragments server

Matches in SemOpenAlex for { <https://semopenalex.org/work/W3100980998> ?p ?o ?g. }

• W3100980998 abstract "Pretrained large-scale language models have increasingly demonstrated high accuracy on many natural language processing (NLP) tasks. However, the limited weight storage and computational speed on hardware platforms have impeded the popularity of pretrained models, especially in the era of edge computing. In this work, we propose an efficient transformer-based large-scale language representation using hardware-friendly block structure pruning. We incorporate the reweighted group Lasso into block-structured pruning for optimization. Besides the significantly reduced weight storage and computation, the proposed approach achieves high compression rates. Experimental results on different models (BERT, RoBERTa, and DistilBERT) on the General Language Understanding Evaluation (GLUE) benchmark tasks show that we achieve up to 5.0x with zero or minor accuracy degradation on certain task(s). Our proposed method is also orthogonal to existing compact pretrained language models such as DistilBERT using knowledge distillation, since a further 1.79x average compression rate can be achieved on top of DistilBERT with zero or minor accuracy degradation. It is suitable to deploy the final compressed model on resource-constrained edge devices." @default.
• W3100980998 created "2020-11-23" @default.
• W3100980998 creator A5013881064 @default.
• W3100980998 creator A5016139042 @default.
• W3100980998 creator A5022756586 @default.
• W3100980998 creator A5030060072 @default.
• W3100980998 creator A5034466262 @default.
• W3100980998 creator A5035730229 @default.
• W3100980998 creator A5078971265 @default.
• W3100980998 date "2020-01-01" @default.
• W3100980998 modified "2023-09-27" @default.
• W3100980998 title "Efficient Transformer-based Large Scale Language Representations using Hardware-friendly Block Structured Pruning" @default.
• W3100980998 cites W131533222 @default.
• W3100980998 cites W1599016936 @default.
• W3100980998 cites W1821462560 @default.
• W3100980998 cites W2043701535 @default.
• W3100980998 cites W2107861471 @default.
• W3100980998 cites W2251939518 @default.
• W3100980998 cites W2285660444 @default.
• W3100980998 cites W2657126969 @default.
• W3100980998 cites W2767785892 @default.
• W3100980998 cites W2787752464 @default.
• W3100980998 cites W2806120502 @default.
• W3100980998 cites W2962739339 @default.
• W3100980998 cites W2963000224 @default.
• W3100980998 cites W2963674932 @default.
• W3100980998 cites W2963748441 @default.
• W3100980998 cites W2963846996 @default.
• W3100980998 cites W2963854351 @default.
• W3100980998 cites W2965373594 @default.
• W3100980998 cites W2970454332 @default.
• W3100980998 cites W2970597249 @default.
• W3100980998 cites W2973727699 @default.
• W3100980998 cites W2978017171 @default.
• W3100980998 cites W2996428491 @default.
• W3100980998 cites W3001800955 @default.
• W3100980998 cites W3104033643 @default.
• W3100980998 cites W3105966348 @default.
• W3100980998 cites W3113747735 @default.
• W3100980998 doi "https://doi.org/10.18653/v1/2020.findings-emnlp.286" @default.
• W3100980998 hasPublicationYear "2020" @default.
• W3100980998 type Work @default.
• W3100980998 sameAs 3100980998 @default.
• W3100980998 citedByCount "11" @default.
• W3100980998 countsByYear W31009809982021 @default.
• W3100980998 countsByYear W31009809982022 @default.
• W3100980998 countsByYear W31009809982023 @default.
• W3100980998 crossrefType "proceedings-article" @default.
• W3100980998 hasAuthorship W3100980998A5013881064 @default.
• W3100980998 hasAuthorship W3100980998A5016139042 @default.
• W3100980998 hasAuthorship W3100980998A5022756586 @default.
• W3100980998 hasAuthorship W3100980998A5030060072 @default.
• W3100980998 hasAuthorship W3100980998A5034466262 @default.
• W3100980998 hasAuthorship W3100980998A5035730229 @default.
• W3100980998 hasAuthorship W3100980998A5078971265 @default.
• W3100980998 hasBestOaLocation W31009809981 @default.
• W3100980998 hasConcept C108010975 @default.
• W3100980998 hasConcept C111919701 @default.
• W3100980998 hasConcept C113775141 @default.
• W3100980998 hasConcept C11413529 @default.
• W3100980998 hasConcept C119857082 @default.
• W3100980998 hasConcept C121332964 @default.
• W3100980998 hasConcept C13280743 @default.
• W3100980998 hasConcept C137293760 @default.
• W3100980998 hasConcept C138236772 @default.
• W3100980998 hasConcept C154945302 @default.
• W3100980998 hasConcept C162307627 @default.
• W3100980998 hasConcept C165801399 @default.
• W3100980998 hasConcept C173608175 @default.
• W3100980998 hasConcept C185798385 @default.
• W3100980998 hasConcept C205649164 @default.
• W3100980998 hasConcept C2524010 @default.
• W3100980998 hasConcept C2777210771 @default.
• W3100980998 hasConcept C33923547 @default.
• W3100980998 hasConcept C41008148 @default.
• W3100980998 hasConcept C45374587 @default.
• W3100980998 hasConcept C62520636 @default.
• W3100980998 hasConcept C6557445 @default.
• W3100980998 hasConcept C66322947 @default.
• W3100980998 hasConcept C79974875 @default.
• W3100980998 hasConcept C86803240 @default.
• W3100980998 hasConceptScore W3100980998C108010975 @default.
• W3100980998 hasConceptScore W3100980998C111919701 @default.
• W3100980998 hasConceptScore W3100980998C113775141 @default.
• W3100980998 hasConceptScore W3100980998C11413529 @default.
• W3100980998 hasConceptScore W3100980998C119857082 @default.
• W3100980998 hasConceptScore W3100980998C121332964 @default.
• W3100980998 hasConceptScore W3100980998C13280743 @default.
• W3100980998 hasConceptScore W3100980998C137293760 @default.
• W3100980998 hasConceptScore W3100980998C138236772 @default.
• W3100980998 hasConceptScore W3100980998C154945302 @default.
• W3100980998 hasConceptScore W3100980998C162307627 @default.
• W3100980998 hasConceptScore W3100980998C165801399 @default.
• W3100980998 hasConceptScore W3100980998C173608175 @default.
• W3100980998 hasConceptScore W3100980998C185798385 @default.
• W3100980998 hasConceptScore W3100980998C205649164 @default.
• W3100980998 hasConceptScore W3100980998C2524010 @default.
• W3100980998 hasConceptScore W3100980998C2777210771 @default.
• W3100980998 hasConceptScore W3100980998C33923547 @default.
• W3100980998 hasConceptScore W3100980998C41008148 @default.

• next