Matches in SemOpenAlex for { <https://semopenalex.org/work/W2783257164> ?p ?o ?g. }
Showing items 1 to 79 of
79
with 100 items per page.
- W2783257164 abstract "As datasets continue to increase in size and multi-core computer architectures are developed, asynchronous parallel optimization algorithms become more and more essential to the field of Machine Learning. Unfortunately, conducting the theoretical analysis asynchronous methods is difficult, notably due to the introduction of delay and inconsistency in inherently sequential algorithms. Handling these issues often requires resorting to simplifying but unrealistic assumptions. Through a novel perspective, we revisit and clarify a subtle but important technical issue present in a large fraction of the recent convergence rate proofs for asynchronous parallel optimization algorithms, and propose a simplification of the recently introduced perturbed iterate framework that resolves it. We demonstrate the usefulness of our new framework by analyzing three distinct asynchronous parallel incremental optimization algorithms: Hogwild (asynchronous SGD), KROMAGNON (asynchronous SVRG) and ASAGA, a novel asynchronous parallel version of the incremental gradient algorithm SAGA that enjoys fast linear convergence rates. We are able to both remove problematic assumptions and obtain better theoretical results. Notably, we prove that ASAGA and KROMAGNON can obtain a theoretical linear speedup on multi-core systems even without sparsity assumptions. We present results of an implementation on a 40-core architecture illustrating the practical speedups as well as the hardware overhead. Finally, we investigate the overlap constant, an ill-understood but central quantity for the theoretical analysis of asynchronous parallel algorithms. We find that it encompasses much more complexity than suggested in previous work, and often is order-of-magnitude bigger than traditionally thought." @default.
- W2783257164 created "2018-01-26" @default.
- W2783257164 creator A5026525021 @default.
- W2783257164 creator A5051076050 @default.
- W2783257164 creator A5066572762 @default.
- W2783257164 date "2018-01-01" @default.
- W2783257164 modified "2023-10-18" @default.
- W2783257164 title "Improved asynchronous parallel optimization analysis for stochastic incremental methods" @default.
- W2783257164 cites W1523661875 @default.
- W2783257164 cites W1603765807 @default.
- W2783257164 cites W1842246424 @default.
- W2783257164 cites W1877037013 @default.
- W2783257164 cites W1939652453 @default.
- W2783257164 cites W1946315329 @default.
- W2783257164 cites W2105875671 @default.
- W2783257164 cites W2107438106 @default.
- W2783257164 cites W2121990650 @default.
- W2783257164 cites W2125126592 @default.
- W2783257164 cites W2135482703 @default.
- W2783257164 cites W2138243089 @default.
- W2783257164 cites W2150102617 @default.
- W2783257164 cites W2156779765 @default.
- W2783257164 cites W2162287622 @default.
- W2783257164 cites W2414514782 @default.
- W2783257164 cites W2616811598 @default.
- W2783257164 cites W2619493178 @default.
- W2783257164 cites W2962798535 @default.
- W2783257164 cites W2962952793 @default.
- W2783257164 cites W2963156201 @default.
- W2783257164 cites W2963615052 @default.
- W2783257164 cites W2963655340 @default.
- W2783257164 cites W2963903325 @default.
- W2783257164 cites W2963965485 @default.
- W2783257164 cites W2964112534 @default.
- W2783257164 cites W2963587506 @default.
- W2783257164 hasPublicationYear "2018" @default.
- W2783257164 type Work @default.
- W2783257164 sameAs 2783257164 @default.
- W2783257164 citedByCount "5" @default.
- W2783257164 countsByYear W27832571642018 @default.
- W2783257164 countsByYear W27832571642019 @default.
- W2783257164 countsByYear W27832571642020 @default.
- W2783257164 crossrefType "journal-article" @default.
- W2783257164 hasAuthorship W2783257164A5026525021 @default.
- W2783257164 hasAuthorship W2783257164A5051076050 @default.
- W2783257164 hasAuthorship W2783257164A5066572762 @default.
- W2783257164 hasBestOaLocation W27832571641 @default.
- W2783257164 hasConcept C126255220 @default.
- W2783257164 hasConcept C151319957 @default.
- W2783257164 hasConcept C173608175 @default.
- W2783257164 hasConcept C194387892 @default.
- W2783257164 hasConcept C31258907 @default.
- W2783257164 hasConcept C33923547 @default.
- W2783257164 hasConcept C41008148 @default.
- W2783257164 hasConceptScore W2783257164C126255220 @default.
- W2783257164 hasConceptScore W2783257164C151319957 @default.
- W2783257164 hasConceptScore W2783257164C173608175 @default.
- W2783257164 hasConceptScore W2783257164C194387892 @default.
- W2783257164 hasConceptScore W2783257164C31258907 @default.
- W2783257164 hasConceptScore W2783257164C33923547 @default.
- W2783257164 hasConceptScore W2783257164C41008148 @default.
- W2783257164 hasLocation W27832571641 @default.
- W2783257164 hasLocation W27832571642 @default.
- W2783257164 hasOpenAccess W2783257164 @default.
- W2783257164 hasPrimaryLocation W27832571641 @default.
- W2783257164 hasRelatedWork W1548344605 @default.
- W2783257164 hasRelatedWork W2006363920 @default.
- W2783257164 hasRelatedWork W2104603305 @default.
- W2783257164 hasRelatedWork W2117014006 @default.
- W2783257164 hasRelatedWork W2122091831 @default.
- W2783257164 hasRelatedWork W2367372208 @default.
- W2783257164 hasRelatedWork W2372170743 @default.
- W2783257164 hasRelatedWork W2475836838 @default.
- W2783257164 hasRelatedWork W3100791781 @default.
- W2783257164 hasRelatedWork W4233815414 @default.
- W2783257164 isParatext "false" @default.
- W2783257164 isRetracted "false" @default.
- W2783257164 magId "2783257164" @default.
- W2783257164 workType "article" @default.