Matches in SemOpenAlex for { <https://semopenalex.org/work/W4293783896> ?p ?o ?g. }
Showing items 1 to 87 of
87
with 100 items per page.
- W4293783896 abstract "In the context of distributed deep learning, the issue of stale weights or gradients could result in poor algorithmic performance. This issue is usually tackled by delay tolerant algorithms with some mild assumptions on the objective functions and step sizes. In this paper, we propose a different approach to develop a new algorithm, called $textbf{P}$redicting $textbf{C}$lipping $textbf{A}$synchronous $textbf{S}$tochastic $textbf{G}$radient $textbf{D}$escent (aka, PC-ASGD). Specifically, PC-ASGD has two steps - the $textit{predicting step}$ leverages the gradient prediction using Taylor expansion to reduce the staleness of the outdated weights while the $textit{clipping step}$ selectively drops the outdated weights to alleviate their negative effects. A tradeoff parameter is introduced to balance the effects between these two steps. Theoretically, we present the convergence rate considering the effects of delay of the proposed algorithm with constant step size when the smooth objective functions are weakly strongly-convex and nonconvex. One practical variant of PC-ASGD is also proposed by adopting a condition to help with the determination of the tradeoff parameter. For empirical validation, we demonstrate the performance of the algorithm with two deep neural network architectures on two benchmark datasets." @default.
- W4293783896 created "2022-08-31" @default.
- W4293783896 creator A5028296024 @default.
- W4293783896 creator A5031212270 @default.
- W4293783896 creator A5042417097 @default.
- W4293783896 creator A5056947172 @default.
- W4293783896 creator A5070006545 @default.
- W4293783896 creator A5081037761 @default.
- W4293783896 date "2022-08-28" @default.
- W4293783896 modified "2023-09-27" @default.
- W4293783896 title "Asynchronous Training Schemes in Distributed Learning with Time Delay" @default.
- W4293783896 doi "https://doi.org/10.48550/arxiv.2208.13154" @default.
- W4293783896 hasPublicationYear "2022" @default.
- W4293783896 type Work @default.
- W4293783896 citedByCount "0" @default.
- W4293783896 crossrefType "posted-content" @default.
- W4293783896 hasAuthorship W4293783896A5028296024 @default.
- W4293783896 hasAuthorship W4293783896A5031212270 @default.
- W4293783896 hasAuthorship W4293783896A5042417097 @default.
- W4293783896 hasAuthorship W4293783896A5056947172 @default.
- W4293783896 hasAuthorship W4293783896A5070006545 @default.
- W4293783896 hasAuthorship W4293783896A5081037761 @default.
- W4293783896 hasBestOaLocation W42937838961 @default.
- W4293783896 hasConcept C112680207 @default.
- W4293783896 hasConcept C11413529 @default.
- W4293783896 hasConcept C127162648 @default.
- W4293783896 hasConcept C13280743 @default.
- W4293783896 hasConcept C138885662 @default.
- W4293783896 hasConcept C145446738 @default.
- W4293783896 hasConcept C151319957 @default.
- W4293783896 hasConcept C151730666 @default.
- W4293783896 hasConcept C162324750 @default.
- W4293783896 hasConcept C185798385 @default.
- W4293783896 hasConcept C199360897 @default.
- W4293783896 hasConcept C205649164 @default.
- W4293783896 hasConcept C2524010 @default.
- W4293783896 hasConcept C2776848632 @default.
- W4293783896 hasConcept C2777027219 @default.
- W4293783896 hasConcept C2777303404 @default.
- W4293783896 hasConcept C2779343474 @default.
- W4293783896 hasConcept C31258907 @default.
- W4293783896 hasConcept C33923547 @default.
- W4293783896 hasConcept C41008148 @default.
- W4293783896 hasConcept C41895202 @default.
- W4293783896 hasConcept C50522688 @default.
- W4293783896 hasConcept C57869625 @default.
- W4293783896 hasConcept C86803240 @default.
- W4293783896 hasConceptScore W4293783896C112680207 @default.
- W4293783896 hasConceptScore W4293783896C11413529 @default.
- W4293783896 hasConceptScore W4293783896C127162648 @default.
- W4293783896 hasConceptScore W4293783896C13280743 @default.
- W4293783896 hasConceptScore W4293783896C138885662 @default.
- W4293783896 hasConceptScore W4293783896C145446738 @default.
- W4293783896 hasConceptScore W4293783896C151319957 @default.
- W4293783896 hasConceptScore W4293783896C151730666 @default.
- W4293783896 hasConceptScore W4293783896C162324750 @default.
- W4293783896 hasConceptScore W4293783896C185798385 @default.
- W4293783896 hasConceptScore W4293783896C199360897 @default.
- W4293783896 hasConceptScore W4293783896C205649164 @default.
- W4293783896 hasConceptScore W4293783896C2524010 @default.
- W4293783896 hasConceptScore W4293783896C2776848632 @default.
- W4293783896 hasConceptScore W4293783896C2777027219 @default.
- W4293783896 hasConceptScore W4293783896C2777303404 @default.
- W4293783896 hasConceptScore W4293783896C2779343474 @default.
- W4293783896 hasConceptScore W4293783896C31258907 @default.
- W4293783896 hasConceptScore W4293783896C33923547 @default.
- W4293783896 hasConceptScore W4293783896C41008148 @default.
- W4293783896 hasConceptScore W4293783896C41895202 @default.
- W4293783896 hasConceptScore W4293783896C50522688 @default.
- W4293783896 hasConceptScore W4293783896C57869625 @default.
- W4293783896 hasConceptScore W4293783896C86803240 @default.
- W4293783896 hasLocation W42937838961 @default.
- W4293783896 hasOpenAccess W4293783896 @default.
- W4293783896 hasPrimaryLocation W42937838961 @default.
- W4293783896 hasRelatedWork W1978709597 @default.
- W4293783896 hasRelatedWork W2021319679 @default.
- W4293783896 hasRelatedWork W2092407260 @default.
- W4293783896 hasRelatedWork W2167914958 @default.
- W4293783896 hasRelatedWork W2350800846 @default.
- W4293783896 hasRelatedWork W2356755074 @default.
- W4293783896 hasRelatedWork W2373948792 @default.
- W4293783896 hasRelatedWork W2387119041 @default.
- W4293783896 hasRelatedWork W2985014567 @default.
- W4293783896 hasRelatedWork W98559547 @default.
- W4293783896 isParatext "false" @default.
- W4293783896 isRetracted "false" @default.
- W4293783896 workType "article" @default.