Matches in SemOpenAlex for { <https://semopenalex.org/work/W4286235879> ?p ?o ?g. }
Showing items 1 to 73 of
73
with 100 items per page.
- W4286235879 abstract "Powered by advances in deep learning (DL) techniques, machine learning and artificial intelligence have achieved astonishing successes. However, the rapidly growing needs for DL also led to communication- and resource-intensive distributed training jobs for large-scale DL training, which are typically deployed over GPU clusters. To sustain the ever-increasing demand for DL training, the so-called ring-all-reduce (RAR) technologies have recently emerged as a favorable computing architecture to efficiently process network communication and computation load in GPU clusters. The most salient feature of RAR is that it removes the need for dedicated parameter servers, thus alleviating the potential communication bottleneck. However, when multiple RAR-based DL training jobs are deployed over GPU clusters, communication bottlenecks could still occur due to contentions between DL training jobs. So far, there remains a lack of theoretical understanding on how to design contention-aware resource scheduling algorithms for RAR-based DL training jobs, which motivates us to fill this gap in this work. Our main contributions are three-fold: i) We develop a new analytical model that characterizes both communication overhead related to the worker distribution of the job and communication contention related to the co-location of different jobs; ii) Based on the proposed analytical model, we formulate the problem as a non-convex integer program to minimize the makespan of all RAR-based DL training jobs. To address the unique structure in this problem that is not amenable for optimization algorithm design, we reformulate the problem into an integer linear program that enables provable approximation algorithm design called SJF-BCO (Smallest Job First with Balanced Contention and Overhead); and iii) We conduct extensive experiments to show the superiority of SJF-BCO over existing schedulers." @default.
- W4286235879 created "2022-07-21" @default.
- W4286235879 creator A5031558801 @default.
- W4286235879 creator A5039054300 @default.
- W4286235879 creator A5057117256 @default.
- W4286235879 creator A5059626072 @default.
- W4286235879 date "2022-07-15" @default.
- W4286235879 modified "2023-09-29" @default.
- W4286235879 title "On Scheduling Ring-All-Reduce Learning Jobs in Multi-Tenant GPU Clusters with Communication Contention" @default.
- W4286235879 doi "https://doi.org/10.48550/arxiv.2207.07817" @default.
- W4286235879 hasPublicationYear "2022" @default.
- W4286235879 type Work @default.
- W4286235879 citedByCount "0" @default.
- W4286235879 crossrefType "posted-content" @default.
- W4286235879 hasAuthorship W4286235879A5031558801 @default.
- W4286235879 hasAuthorship W4286235879A5039054300 @default.
- W4286235879 hasAuthorship W4286235879A5057117256 @default.
- W4286235879 hasAuthorship W4286235879A5059626072 @default.
- W4286235879 hasBestOaLocation W42862358791 @default.
- W4286235879 hasConcept C111919701 @default.
- W4286235879 hasConcept C11413529 @default.
- W4286235879 hasConcept C119857082 @default.
- W4286235879 hasConcept C120314980 @default.
- W4286235879 hasConcept C126255220 @default.
- W4286235879 hasConcept C149635348 @default.
- W4286235879 hasConcept C154945302 @default.
- W4286235879 hasConcept C173608175 @default.
- W4286235879 hasConcept C206729178 @default.
- W4286235879 hasConcept C2779960059 @default.
- W4286235879 hasConcept C2780513914 @default.
- W4286235879 hasConcept C2781335571 @default.
- W4286235879 hasConcept C31258907 @default.
- W4286235879 hasConcept C33923547 @default.
- W4286235879 hasConcept C41008148 @default.
- W4286235879 hasConcept C45374587 @default.
- W4286235879 hasConcept C55416958 @default.
- W4286235879 hasConcept C74172769 @default.
- W4286235879 hasConcept C93996380 @default.
- W4286235879 hasConceptScore W4286235879C111919701 @default.
- W4286235879 hasConceptScore W4286235879C11413529 @default.
- W4286235879 hasConceptScore W4286235879C119857082 @default.
- W4286235879 hasConceptScore W4286235879C120314980 @default.
- W4286235879 hasConceptScore W4286235879C126255220 @default.
- W4286235879 hasConceptScore W4286235879C149635348 @default.
- W4286235879 hasConceptScore W4286235879C154945302 @default.
- W4286235879 hasConceptScore W4286235879C173608175 @default.
- W4286235879 hasConceptScore W4286235879C206729178 @default.
- W4286235879 hasConceptScore W4286235879C2779960059 @default.
- W4286235879 hasConceptScore W4286235879C2780513914 @default.
- W4286235879 hasConceptScore W4286235879C2781335571 @default.
- W4286235879 hasConceptScore W4286235879C31258907 @default.
- W4286235879 hasConceptScore W4286235879C33923547 @default.
- W4286235879 hasConceptScore W4286235879C41008148 @default.
- W4286235879 hasConceptScore W4286235879C45374587 @default.
- W4286235879 hasConceptScore W4286235879C55416958 @default.
- W4286235879 hasConceptScore W4286235879C74172769 @default.
- W4286235879 hasConceptScore W4286235879C93996380 @default.
- W4286235879 hasLocation W42862358791 @default.
- W4286235879 hasOpenAccess W4286235879 @default.
- W4286235879 hasPrimaryLocation W42862358791 @default.
- W4286235879 hasRelatedWork W1493993314 @default.
- W4286235879 hasRelatedWork W1513409726 @default.
- W4286235879 hasRelatedWork W1597592026 @default.
- W4286235879 hasRelatedWork W2170649215 @default.
- W4286235879 hasRelatedWork W2952511314 @default.
- W4286235879 hasRelatedWork W4221149636 @default.
- W4286235879 hasRelatedWork W4286235879 @default.
- W4286235879 hasRelatedWork W4287929077 @default.
- W4286235879 hasRelatedWork W94000989 @default.
- W4286235879 hasRelatedWork W2516240846 @default.
- W4286235879 isParatext "false" @default.
- W4286235879 isRetracted "false" @default.
- W4286235879 workType "article" @default.