Matches in SemOpenAlex for { <https://semopenalex.org/work/W4384652344> ?p ?o ?g. }
Showing items 1 to 75 of
75
with 100 items per page.
- W4384652344 abstract "Two widely considered decentralized learning algorithms are Gossip and random walk-based learning. Gossip algorithms (both synchronous and asynchronous versions) suffer from high communication cost, while random-walk based learning experiences increased convergence time. In this paper, we design a fast and communication-efficient asynchronous decentralized learning mechanism DIGEST by taking advantage of both Gossip and random-walk ideas, and focusing on stochastic gradient descent (SGD). DIGEST is an asynchronous decentralized algorithm building on local-SGD algorithms, which are originally designed for communication efficient centralized learning. We design both single-stream and multi-stream DIGEST, where the communication overhead may increase when the number of streams increases, and there is a convergence and communication overhead trade-off which can be leveraged. We analyze the convergence of single- and multi-stream DIGEST, and prove that both algorithms approach to the optimal solution asymptotically for both iid and non-iid data distributions. We evaluate the performance of single- and multi-stream DIGEST for logistic regression and a deep neural network ResNet20. The simulation results confirm that multi-stream DIGEST has nice convergence properties; i.e., its convergence time is better than or comparable to the baselines in iid setting, and outperforms the baselines in non-iid setting." @default.
- W4384652344 created "2023-07-19" @default.
- W4384652344 creator A5059833598 @default.
- W4384652344 creator A5061388526 @default.
- W4384652344 date "2023-07-14" @default.
- W4384652344 modified "2023-10-18" @default.
- W4384652344 title "DIGEST: Fast and Communication Efficient Decentralized Learning with Local Updates" @default.
- W4384652344 doi "https://doi.org/10.48550/arxiv.2307.07652" @default.
- W4384652344 hasPublicationYear "2023" @default.
- W4384652344 type Work @default.
- W4384652344 citedByCount "0" @default.
- W4384652344 crossrefType "posted-content" @default.
- W4384652344 hasAuthorship W4384652344A5059833598 @default.
- W4384652344 hasAuthorship W4384652344A5061388526 @default.
- W4384652344 hasBestOaLocation W43846523441 @default.
- W4384652344 hasConcept C105795698 @default.
- W4384652344 hasConcept C111919701 @default.
- W4384652344 hasConcept C11413529 @default.
- W4384652344 hasConcept C119857082 @default.
- W4384652344 hasConcept C120314980 @default.
- W4384652344 hasConcept C121194460 @default.
- W4384652344 hasConcept C151319957 @default.
- W4384652344 hasConcept C154945302 @default.
- W4384652344 hasConcept C15744967 @default.
- W4384652344 hasConcept C162324750 @default.
- W4384652344 hasConcept C206688291 @default.
- W4384652344 hasConcept C2777303404 @default.
- W4384652344 hasConcept C2778484313 @default.
- W4384652344 hasConcept C2779960059 @default.
- W4384652344 hasConcept C31258907 @default.
- W4384652344 hasConcept C33923547 @default.
- W4384652344 hasConcept C41008148 @default.
- W4384652344 hasConcept C50522688 @default.
- W4384652344 hasConcept C50644808 @default.
- W4384652344 hasConcept C62989814 @default.
- W4384652344 hasConcept C76155785 @default.
- W4384652344 hasConcept C77805123 @default.
- W4384652344 hasConceptScore W4384652344C105795698 @default.
- W4384652344 hasConceptScore W4384652344C111919701 @default.
- W4384652344 hasConceptScore W4384652344C11413529 @default.
- W4384652344 hasConceptScore W4384652344C119857082 @default.
- W4384652344 hasConceptScore W4384652344C120314980 @default.
- W4384652344 hasConceptScore W4384652344C121194460 @default.
- W4384652344 hasConceptScore W4384652344C151319957 @default.
- W4384652344 hasConceptScore W4384652344C154945302 @default.
- W4384652344 hasConceptScore W4384652344C15744967 @default.
- W4384652344 hasConceptScore W4384652344C162324750 @default.
- W4384652344 hasConceptScore W4384652344C206688291 @default.
- W4384652344 hasConceptScore W4384652344C2777303404 @default.
- W4384652344 hasConceptScore W4384652344C2778484313 @default.
- W4384652344 hasConceptScore W4384652344C2779960059 @default.
- W4384652344 hasConceptScore W4384652344C31258907 @default.
- W4384652344 hasConceptScore W4384652344C33923547 @default.
- W4384652344 hasConceptScore W4384652344C41008148 @default.
- W4384652344 hasConceptScore W4384652344C50522688 @default.
- W4384652344 hasConceptScore W4384652344C50644808 @default.
- W4384652344 hasConceptScore W4384652344C62989814 @default.
- W4384652344 hasConceptScore W4384652344C76155785 @default.
- W4384652344 hasConceptScore W4384652344C77805123 @default.
- W4384652344 hasLocation W43846523441 @default.
- W4384652344 hasOpenAccess W4384652344 @default.
- W4384652344 hasPrimaryLocation W43846523441 @default.
- W4384652344 hasRelatedWork W123872086 @default.
- W4384652344 hasRelatedWork W1561560534 @default.
- W4384652344 hasRelatedWork W1994599362 @default.
- W4384652344 hasRelatedWork W2092071486 @default.
- W4384652344 hasRelatedWork W2313989154 @default.
- W4384652344 hasRelatedWork W2364921833 @default.
- W4384652344 hasRelatedWork W2391167130 @default.
- W4384652344 hasRelatedWork W2558946687 @default.
- W4384652344 hasRelatedWork W4300427724 @default.
- W4384652344 hasRelatedWork W4316171247 @default.
- W4384652344 isParatext "false" @default.
- W4384652344 isRetracted "false" @default.
- W4384652344 workType "article" @default.