FRINK Linked Data Fragments server

Matches in SemOpenAlex for { <https://semopenalex.org/work/W3105148963> ?p ?o ?g. }

• W3105148963 endingPage "13563" @default.
• W3105148963 startingPage "13551" @default.
• W3105148963 abstract "Large-scale distributed training of Deep Neural Networks (DNNs) on state-of-the-art platforms is expected to be severely communication constrained. To overcome this limitation, numerous gradient compression techniques have been proposed and have demonstrated high compression ratios. However, most existing methods do not scale well to large scale distributed systems (due to gradient build-up) and/or fail to evaluate model fidelity (test accuracy) on large datasets. To mitigate these issues, we propose a new compression technique, Scalable Sparsified Gradient Compression (ScaleCom), that leverages similarity in the gradient distribution amongst learners to provide significantly improved scalability. Using theoretical analysis, we show that ScaleCom provides favorable convergence guarantees and is compatible with gradient all-reduce techniques. Furthermore, we experimentally demonstrate that ScaleCom has small overheads, directly reduces gradient traffic and provides high compression rates (65-400X) and excellent scalability (up to 64 learners and 8-12X larger batch sizes over standard training) across a wide range of applications (image, language, and speech) without significant accuracy loss." @default.
• W3105148963 created "2020-11-23" @default.
• W3105148963 creator A5001435691 @default.
• W3105148963 creator A5004766079 @default.
• W3105148963 creator A5010094713 @default.
• W3105148963 creator A5018198614 @default.
• W3105148963 creator A5041602011 @default.
• W3105148963 creator A5050344371 @default.
• W3105148963 creator A5056502880 @default.
• W3105148963 creator A5076699095 @default.
• W3105148963 creator A5082043392 @default.
• W3105148963 creator A5088593720 @default.
• W3105148963 creator A5089852582 @default.
• W3105148963 date "2020-01-01" @default.
• W3105148963 modified "2023-09-23" @default.
• W3105148963 title "ScaleCom: Scalable Sparsified Gradient Compression for Communication-Efficient Distributed Training" @default.
• W3105148963 hasPublicationYear "2020" @default.
• W3105148963 type Work @default.
• W3105148963 sameAs 3105148963 @default.
• W3105148963 citedByCount "3" @default.
• W3105148963 countsByYear W31051489632020 @default.
• W3105148963 countsByYear W31051489632021 @default.
• W3105148963 crossrefType "proceedings-article" @default.
• W3105148963 hasAuthorship W3105148963A5001435691 @default.
• W3105148963 hasAuthorship W3105148963A5004766079 @default.
• W3105148963 hasAuthorship W3105148963A5010094713 @default.
• W3105148963 hasAuthorship W3105148963A5018198614 @default.
• W3105148963 hasAuthorship W3105148963A5041602011 @default.
• W3105148963 hasAuthorship W3105148963A5050344371 @default.
• W3105148963 hasAuthorship W3105148963A5056502880 @default.
• W3105148963 hasAuthorship W3105148963A5076699095 @default.
• W3105148963 hasAuthorship W3105148963A5082043392 @default.
• W3105148963 hasAuthorship W3105148963A5088593720 @default.
• W3105148963 hasAuthorship W3105148963A5089852582 @default.
• W3105148963 hasConcept C113364801 @default.
• W3105148963 hasConcept C113775141 @default.
• W3105148963 hasConcept C119599485 @default.
• W3105148963 hasConcept C120314980 @default.
• W3105148963 hasConcept C121332964 @default.
• W3105148963 hasConcept C127413603 @default.
• W3105148963 hasConcept C154945302 @default.
• W3105148963 hasConcept C159985019 @default.
• W3105148963 hasConcept C162324750 @default.
• W3105148963 hasConcept C171146098 @default.
• W3105148963 hasConcept C180016635 @default.
• W3105148963 hasConcept C192562407 @default.
• W3105148963 hasConcept C25797200 @default.
• W3105148963 hasConcept C2777303404 @default.
• W3105148963 hasConcept C2778755073 @default.
• W3105148963 hasConcept C41008148 @default.
• W3105148963 hasConcept C48044578 @default.
• W3105148963 hasConcept C50522688 @default.
• W3105148963 hasConcept C511840579 @default.
• W3105148963 hasConcept C62520636 @default.
• W3105148963 hasConcept C77088390 @default.
• W3105148963 hasConceptScore W3105148963C113364801 @default.
• W3105148963 hasConceptScore W3105148963C113775141 @default.
• W3105148963 hasConceptScore W3105148963C119599485 @default.
• W3105148963 hasConceptScore W3105148963C120314980 @default.
• W3105148963 hasConceptScore W3105148963C121332964 @default.
• W3105148963 hasConceptScore W3105148963C127413603 @default.
• W3105148963 hasConceptScore W3105148963C154945302 @default.
• W3105148963 hasConceptScore W3105148963C159985019 @default.
• W3105148963 hasConceptScore W3105148963C162324750 @default.
• W3105148963 hasConceptScore W3105148963C171146098 @default.
• W3105148963 hasConceptScore W3105148963C180016635 @default.
• W3105148963 hasConceptScore W3105148963C192562407 @default.
• W3105148963 hasConceptScore W3105148963C25797200 @default.
• W3105148963 hasConceptScore W3105148963C2777303404 @default.
• W3105148963 hasConceptScore W3105148963C2778755073 @default.
• W3105148963 hasConceptScore W3105148963C41008148 @default.
• W3105148963 hasConceptScore W3105148963C48044578 @default.
• W3105148963 hasConceptScore W3105148963C50522688 @default.
• W3105148963 hasConceptScore W3105148963C511840579 @default.
• W3105148963 hasConceptScore W3105148963C62520636 @default.
• W3105148963 hasConceptScore W3105148963C77088390 @default.
• W3105148963 hasLocation W31051489631 @default.
• W3105148963 hasOpenAccess W3105148963 @default.
• W3105148963 hasPrimaryLocation W31051489631 @default.
• W3105148963 hasRelatedWork W2617019596 @default.
• W3105148963 hasRelatedWork W2617766261 @default.
• W3105148963 hasRelatedWork W2769644379 @default.
• W3105148963 hasRelatedWork W2901028208 @default.
• W3105148963 hasRelatedWork W2962710991 @default.
• W3105148963 hasRelatedWork W3028027935 @default.
• W3105148963 hasRelatedWork W3035605662 @default.
• W3105148963 hasRelatedWork W3048804513 @default.
• W3105148963 hasRelatedWork W3087165870 @default.
• W3105148963 hasRelatedWork W3096004388 @default.
• W3105148963 hasRelatedWork W3100246930 @default.
• W3105148963 hasRelatedWork W3104789770 @default.
• W3105148963 hasRelatedWork W3131685502 @default.
• W3105148963 hasRelatedWork W3154560182 @default.
• W3105148963 hasRelatedWork W3174871729 @default.
• W3105148963 hasRelatedWork W3198707348 @default.
• W3105148963 hasRelatedWork W3204450046 @default.
• W3105148963 hasRelatedWork W3204507145 @default.
• W3105148963 hasRelatedWork W3205035978 @default.

• next