Matches in SemOpenAlex for { <https://semopenalex.org/work/W3174111838> ?p ?o ?g. }
Showing items 1 to 82 of
82
with 100 items per page.
- W3174111838 abstract "Distributed Execution Frameworks (DEFs) such as Apache Spark have become ubiquitous as a solution for the execution of user-defined jobs to process terabytes of data across hundreds of nodes. One of the key costs of DEFs is scheduling of which parts of each job are placed on each host; better scheduling decisions provide lower overall execution time for each job, more efficient resource usage, and reduced energy consumption. Existing DEFs use a static approach to scheduling, either with a single generalised scheduler which aims to be a good fit for most workloads, or with a special-purpose scheduler which is tuned to optimise for a particular kind of workload. In both cases the scheduling implementation is fixed at design-time such that the DEF is unable to adjust to the actual characteristics of workloads that arrive at deployment time. In this paper we introduce an emergent scheduler for Distributed Execution Frameworks. This scheduler can be composed and re-composed at runtime from a set of different building blocks, allowing the system to dynamically provide the benefits of differing scheduling policies over time depending on the actual properties of incoming workloads - with improved performance and resource usage. In this paper we present the overall design of our emergent scheduler, we discuss the theoretical design space of different scheduling approaches, and we examine a specific research question to determine the correlation between workload properties and scheduling performance for different scheduler implementations. Our results are based on a real implementation of our emergent DEF running across multiple hosts in a real datacentre, and our implementation is made available as open-source software." @default.
- W3174111838 created "2021-07-05" @default.
- W3174111838 creator A5079160321 @default.
- W3174111838 creator A5079859307 @default.
- W3174111838 date "2021-05-01" @default.
- W3174111838 modified "2023-09-27" @default.
- W3174111838 title "The Design Space of Emergent Scheduling for Distributed Execution Frameworks" @default.
- W3174111838 cites W185167974 @default.
- W3174111838 cites W2022678927 @default.
- W3174111838 cites W2036234282 @default.
- W3174111838 cites W2067073559 @default.
- W3174111838 cites W2096125134 @default.
- W3174111838 cites W2102849319 @default.
- W3174111838 cites W2105947650 @default.
- W3174111838 cites W2559241810 @default.
- W3174111838 cites W2560637284 @default.
- W3174111838 cites W2766032408 @default.
- W3174111838 cites W2769821579 @default.
- W3174111838 cites W2798434712 @default.
- W3174111838 cites W2913854892 @default.
- W3174111838 cites W2943010996 @default.
- W3174111838 cites W2968986602 @default.
- W3174111838 cites W2990017454 @default.
- W3174111838 cites W4253824360 @default.
- W3174111838 doi "https://doi.org/10.1109/seams51251.2021.00032" @default.
- W3174111838 hasPublicationYear "2021" @default.
- W3174111838 type Work @default.
- W3174111838 sameAs 3174111838 @default.
- W3174111838 citedByCount "6" @default.
- W3174111838 countsByYear W31741118382021 @default.
- W3174111838 countsByYear W31741118382022 @default.
- W3174111838 countsByYear W31741118382023 @default.
- W3174111838 crossrefType "proceedings-article" @default.
- W3174111838 hasAuthorship W3174111838A5079160321 @default.
- W3174111838 hasAuthorship W3174111838A5079859307 @default.
- W3174111838 hasBestOaLocation W31741118382 @default.
- W3174111838 hasConcept C105339364 @default.
- W3174111838 hasConcept C107568181 @default.
- W3174111838 hasConcept C111919701 @default.
- W3174111838 hasConcept C115903868 @default.
- W3174111838 hasConcept C120314980 @default.
- W3174111838 hasConcept C122141398 @default.
- W3174111838 hasConcept C127456818 @default.
- W3174111838 hasConcept C162324750 @default.
- W3174111838 hasConcept C206729178 @default.
- W3174111838 hasConcept C21547014 @default.
- W3174111838 hasConcept C26713055 @default.
- W3174111838 hasConcept C2778476105 @default.
- W3174111838 hasConcept C41008148 @default.
- W3174111838 hasConcept C68387754 @default.
- W3174111838 hasConceptScore W3174111838C105339364 @default.
- W3174111838 hasConceptScore W3174111838C107568181 @default.
- W3174111838 hasConceptScore W3174111838C111919701 @default.
- W3174111838 hasConceptScore W3174111838C115903868 @default.
- W3174111838 hasConceptScore W3174111838C120314980 @default.
- W3174111838 hasConceptScore W3174111838C122141398 @default.
- W3174111838 hasConceptScore W3174111838C127456818 @default.
- W3174111838 hasConceptScore W3174111838C162324750 @default.
- W3174111838 hasConceptScore W3174111838C206729178 @default.
- W3174111838 hasConceptScore W3174111838C21547014 @default.
- W3174111838 hasConceptScore W3174111838C26713055 @default.
- W3174111838 hasConceptScore W3174111838C2778476105 @default.
- W3174111838 hasConceptScore W3174111838C41008148 @default.
- W3174111838 hasConceptScore W3174111838C68387754 @default.
- W3174111838 hasLocation W31741118381 @default.
- W3174111838 hasLocation W31741118382 @default.
- W3174111838 hasOpenAccess W3174111838 @default.
- W3174111838 hasPrimaryLocation W31741118381 @default.
- W3174111838 hasRelatedWork W1507544503 @default.
- W3174111838 hasRelatedWork W1542006692 @default.
- W3174111838 hasRelatedWork W2017416991 @default.
- W3174111838 hasRelatedWork W2098793920 @default.
- W3174111838 hasRelatedWork W2100233956 @default.
- W3174111838 hasRelatedWork W2184392173 @default.
- W3174111838 hasRelatedWork W2364069560 @default.
- W3174111838 hasRelatedWork W2731842272 @default.
- W3174111838 hasRelatedWork W2978454856 @default.
- W3174111838 hasRelatedWork W3174111838 @default.
- W3174111838 isParatext "false" @default.
- W3174111838 isRetracted "false" @default.
- W3174111838 magId "3174111838" @default.
- W3174111838 workType "article" @default.