FRINK Linked Data Fragments server

Matches in SemOpenAlex for { <https://semopenalex.org/work/W3204188483> ?p ?o ?g. }

• W3204188483 abstract "We study the MARINA method of Gorbunov et al (2021) -- the current state-of-the-art distributed non-convex optimization method in terms of theoretical communication complexity. Theoretical superiority of this method can be largely attributed to two sources: the use of a carefully engineered biased stochastic gradient estimator, which leads to a reduction in the number of communication rounds, and the reliance on {em independent} stochastic communication compression operators, which leads to a reduction in the number of transmitted bits within each communication round. In this paper we i) extend the theory of MARINA to support a much wider class of potentially {em correlated} compressors, extending the reach of the method beyond the classical independent compressors setting, ii) show that a new quantity, for which we coin the name {em Hessian variance}, allows us to significantly refine the original analysis of MARINA without any additional assumptions, and iii) identify a special class of correlated compressors based on the idea of {em random permutations}, for which we coin the term Perm$K$, the use of which leads to $O(sqrt{n})$ (resp. $O(1 + d/sqrt{n})$) improvement in the theoretical communication complexity of MARINA in the low Hessian variance regime when $dgeq n$ (resp. $d leq n$), where $n$ is the number of workers and $d$ is the number of parameters describing the model we are learning. We corroborate our theoretical results with carefully engineered synthetic experiments with minimizing the average of nonconvex quadratics, and on autoencoder training with the MNIST dataset." @default.
• W3204188483 created "2021-10-11" @default.
• W3204188483 creator A5023853765 @default.
• W3204188483 creator A5036598221 @default.
• W3204188483 creator A5080650528 @default.
• W3204188483 date "2021-10-07" @default.
• W3204188483 modified "2023-09-26" @default.
• W3204188483 title "Permutation Compressors for Provably Faster Distributed Nonconvex Optimization" @default.
• W3204188483 cites W1981663184 @default.
• W3204188483 cites W2407022425 @default.
• W3204188483 cites W2556846208 @default.
• W3204188483 cites W2769644379 @default.
• W3204188483 cites W2808495300 @default.
• W3204188483 cites W2890924858 @default.
• W3204188483 cites W2911327153 @default.
• W3204188483 cites W2911863041 @default.
• W3204188483 cites W2914138317 @default.
• W3204188483 cites W2946198813 @default.
• W3204188483 cites W2946815927 @default.
• W3204188483 cites W2954365017 @default.
• W3204188483 cites W2962696654 @default.
• W3204188483 cites W2963403868 @default.
• W3204188483 cites W2963411541 @default.
• W3204188483 cites W2963766684 @default.
• W3204188483 cites W2964163156 @default.
• W3204188483 cites W2970289928 @default.
• W3204188483 cites W2973096819 @default.
• W3204188483 cites W3005128411 @default.
• W3204188483 cites W3007528421 @default.
• W3204188483 cites W3037842861 @default.
• W3204188483 cites W3080910898 @default.
• W3204188483 cites W3098903812 @default.
• W3204188483 cites W3100770245 @default.
• W3204188483 cites W3113984671 @default.
• W3204188483 cites W3133646186 @default.
• W3204188483 cites W3166972615 @default.
• W3204188483 cites W3171765558 @default.
• W3204188483 hasPublicationYear "2021" @default.
• W3204188483 type Work @default.
• W3204188483 sameAs 3204188483 @default.
• W3204188483 citedByCount "0" @default.
• W3204188483 crossrefType "posted-content" @default.
• W3204188483 hasAuthorship W3204188483A5023853765 @default.
• W3204188483 hasAuthorship W3204188483A5036598221 @default.
• W3204188483 hasAuthorship W3204188483A5080650528 @default.
• W3204188483 hasConcept C105795698 @default.
• W3204188483 hasConcept C108583219 @default.
• W3204188483 hasConcept C111335779 @default.
• W3204188483 hasConcept C112680207 @default.
• W3204188483 hasConcept C11413529 @default.
• W3204188483 hasConcept C114614502 @default.
• W3204188483 hasConcept C115903868 @default.
• W3204188483 hasConcept C118615104 @default.
• W3204188483 hasConcept C121332964 @default.
• W3204188483 hasConcept C121955636 @default.
• W3204188483 hasConcept C126255220 @default.
• W3204188483 hasConcept C138885662 @default.
• W3204188483 hasConcept C144133560 @default.
• W3204188483 hasConcept C154945302 @default.
• W3204188483 hasConcept C190502265 @default.
• W3204188483 hasConcept C19499675 @default.
• W3204188483 hasConcept C196083921 @default.
• W3204188483 hasConcept C203616005 @default.
• W3204188483 hasConcept C207390915 @default.
• W3204188483 hasConcept C21308566 @default.
• W3204188483 hasConcept C24890656 @default.
• W3204188483 hasConcept C2524010 @default.
• W3204188483 hasConcept C2777212361 @default.
• W3204188483 hasConcept C28826006 @default.
• W3204188483 hasConcept C33923547 @default.
• W3204188483 hasConcept C41008148 @default.
• W3204188483 hasConcept C41895202 @default.
• W3204188483 hasConcept C55166926 @default.
• W3204188483 hasConcept C62644790 @default.
• W3204188483 hasConceptScore W3204188483C105795698 @default.
• W3204188483 hasConceptScore W3204188483C108583219 @default.
• W3204188483 hasConceptScore W3204188483C111335779 @default.
• W3204188483 hasConceptScore W3204188483C112680207 @default.
• W3204188483 hasConceptScore W3204188483C11413529 @default.
• W3204188483 hasConceptScore W3204188483C114614502 @default.
• W3204188483 hasConceptScore W3204188483C115903868 @default.
• W3204188483 hasConceptScore W3204188483C118615104 @default.
• W3204188483 hasConceptScore W3204188483C121332964 @default.
• W3204188483 hasConceptScore W3204188483C121955636 @default.
• W3204188483 hasConceptScore W3204188483C126255220 @default.
• W3204188483 hasConceptScore W3204188483C138885662 @default.
• W3204188483 hasConceptScore W3204188483C144133560 @default.
• W3204188483 hasConceptScore W3204188483C154945302 @default.
• W3204188483 hasConceptScore W3204188483C190502265 @default.
• W3204188483 hasConceptScore W3204188483C19499675 @default.
• W3204188483 hasConceptScore W3204188483C196083921 @default.
• W3204188483 hasConceptScore W3204188483C203616005 @default.
• W3204188483 hasConceptScore W3204188483C207390915 @default.
• W3204188483 hasConceptScore W3204188483C21308566 @default.
• W3204188483 hasConceptScore W3204188483C24890656 @default.
• W3204188483 hasConceptScore W3204188483C2524010 @default.
• W3204188483 hasConceptScore W3204188483C2777212361 @default.
• W3204188483 hasConceptScore W3204188483C28826006 @default.
• W3204188483 hasConceptScore W3204188483C33923547 @default.
• W3204188483 hasConceptScore W3204188483C41008148 @default.

• next