SemOpenAlex |

SemOpenAlex

Matches in SemOpenAlex for { <https://semopenalex.org/work/W2952714342> ?p ?o ?g. }

Showing items 1 to 100 of ±119 with 100 items per page.

W2952714342 abstract "For SGD based distributed stochastic optimization, computation complexity, measured by the convergence rate in terms of the number of stochastic gradient calls, and communication complexity, measured by the number of inter-node communication rounds, are two most important performance metrics. The classical data-parallel implementation of SGD over $N$ workers can achieve linear speedup of its convergence rate but incurs an inter-node communication round at each batch. We study the benefit of using dynamically increasing batch sizes in parallel SGD for stochastic non-convex optimization by charactering the attained convergence rate and the required number of communication rounds. We show that for stochastic non-convex optimization under the P-L condition, the classical data-parallel SGD with exponentially increasing batch sizes can achieve the fastest known $O(1/(NT))$ convergence with linear speedup using only $log(T)$ communication rounds. For general stochastic non-convex optimization, we propose a Catalyst-like algorithm to achieve the fastest known $O(1/sqrt{NT})$ convergence with only $O(sqrt{NT}log(frac{T}{N}))$ communication rounds." @default.
W2952714342 created "2019-06-27" @default.
W2952714342 creator A5055267677 @default.
W2952714342 creator A5069394608 @default.
W2952714342 date "2019-05-10" @default.
W2952714342 modified "2023-09-27" @default.
W2952714342 title "On the Computation and Communication Complexity of Parallel SGD with Dynamic Batch Sizes for Stochastic Non-Convex Optimization." @default.
W2952714342 cites W1485418074 @default.
W2952714342 cites W1505731132 @default.
W2952714342 cites W1992208280 @default.
W2952714342 cites W2029463628 @default.
W2952714342 cites W2037679050 @default.
W2952714342 cites W2079491734 @default.
W2952714342 cites W2112269233 @default.
W2952714342 cites W2113651538 @default.
W2952714342 cites W2130062883 @default.
W2952714342 cites W2151695970 @default.
W2952714342 cites W2154682027 @default.
W2952714342 cites W2194775991 @default.
W2952714342 cites W2622379787 @default.
W2952714342 cites W2735159666 @default.
W2952714342 cites W2767674840 @default.
W2952714342 cites W2773689216 @default.
W2952714342 cites W2798733011 @default.
W2952714342 cites W2803867449 @default.
W2952714342 cites W2888206291 @default.
W2952714342 cites W2888561381 @default.
W2952714342 cites W2891735383 @default.
W2952714342 cites W2951862792 @default.
W2952714342 cites W2963248893 @default.
W2952714342 cites W2963433607 @default.
W2952714342 cites W2963470657 @default.
W2952714342 cites W2963576077 @default.
W2952714342 cites W2963702144 @default.
W2952714342 cites W2963903325 @default.
W2952714342 cites W3118608800 @default.
W2952714342 cites W3141595720 @default.
W2952714342 cites W3183134511 @default.
W2952714342 cites W584312462 @default.
W2952714342 hasPublicationYear "2019" @default.
W2952714342 type Work @default.
W2952714342 sameAs 2952714342 @default.
W2952714342 citedByCount "3" @default.
W2952714342 countsByYear W29527143422020 @default.
W2952714342 countsByYear W29527143422021 @default.
W2952714342 crossrefType "posted-content" @default.
W2952714342 hasAuthorship W2952714342A5055267677 @default.
W2952714342 hasAuthorship W2952714342A5069394608 @default.
W2952714342 hasConcept C112680207 @default.
W2952714342 hasConcept C11413529 @default.
W2952714342 hasConcept C121332964 @default.
W2952714342 hasConcept C126255220 @default.
W2952714342 hasConcept C127162648 @default.
W2952714342 hasConcept C145446738 @default.
W2952714342 hasConcept C157972887 @default.
W2952714342 hasConcept C162324750 @default.
W2952714342 hasConcept C173608175 @default.
W2952714342 hasConcept C179145077 @default.
W2952714342 hasConcept C194387892 @default.
W2952714342 hasConcept C2524010 @default.
W2952714342 hasConcept C2777303404 @default.
W2952714342 hasConcept C33923547 @default.
W2952714342 hasConcept C41008148 @default.
W2952714342 hasConcept C45374587 @default.
W2952714342 hasConcept C50522688 @default.
W2952714342 hasConcept C57869625 @default.
W2952714342 hasConcept C62520636 @default.
W2952714342 hasConcept C62611344 @default.
W2952714342 hasConcept C68339613 @default.
W2952714342 hasConcept C76155785 @default.
W2952714342 hasConceptScore W2952714342C112680207 @default.
W2952714342 hasConceptScore W2952714342C11413529 @default.
W2952714342 hasConceptScore W2952714342C121332964 @default.
W2952714342 hasConceptScore W2952714342C126255220 @default.
W2952714342 hasConceptScore W2952714342C127162648 @default.
W2952714342 hasConceptScore W2952714342C145446738 @default.
W2952714342 hasConceptScore W2952714342C157972887 @default.
W2952714342 hasConceptScore W2952714342C162324750 @default.
W2952714342 hasConceptScore W2952714342C173608175 @default.
W2952714342 hasConceptScore W2952714342C179145077 @default.
W2952714342 hasConceptScore W2952714342C194387892 @default.
W2952714342 hasConceptScore W2952714342C2524010 @default.
W2952714342 hasConceptScore W2952714342C2777303404 @default.
W2952714342 hasConceptScore W2952714342C33923547 @default.
W2952714342 hasConceptScore W2952714342C41008148 @default.
W2952714342 hasConceptScore W2952714342C45374587 @default.
W2952714342 hasConceptScore W2952714342C50522688 @default.
W2952714342 hasConceptScore W2952714342C57869625 @default.
W2952714342 hasConceptScore W2952714342C62520636 @default.
W2952714342 hasConceptScore W2952714342C62611344 @default.
W2952714342 hasConceptScore W2952714342C68339613 @default.
W2952714342 hasConceptScore W2952714342C76155785 @default.
W2952714342 hasLocation W29527143421 @default.
W2952714342 hasOpenAccess W2952714342 @default.
W2952714342 hasPrimaryLocation W29527143421 @default.
W2952714342 hasRelatedWork W1606909871 @default.
W2952714342 hasRelatedWork W2168803043 @default.
W2952714342 hasRelatedWork W2223328134 @default.
W2952714342 hasRelatedWork W2229196312 @default.
W2952714342 hasRelatedWork W2546267725 @default.