Matches in SemOpenAlex for { <https://semopenalex.org/work/W3129353670> ?p ?o ?g. }
- W3129353670 endingPage "530" @default.
- W3129353670 startingPage "519" @default.
- W3129353670 abstract "Distributed data processing systems like MapReduce, Spark, and Flink are popular tools for analysis of large datasets with cluster resources. Yet, users often overprovision resources for their data processing jobs, while the resource usage of these jobs also typically fluctuates considerably. Therefore, multiple jobs usually get scheduled onto the same shared resources to increase the resource utilization and throughput of clusters. However, job runtimes and the utilization of shared resources can vary significantly depending on the specific combinations of co-located jobs. This paper presents Hugo, a cluster scheduler that continuously learns how efficiently jobs share resources, considering metrics for the resource utilization and interference among co-located jobs. The scheduler combines offline grouping of jobs with online reinforcement learning to provide a scheduling mechanism that efficiently generalizes from specific monitored job combinations yet also adapts to changes in workloads. Our evaluation of a prototype shows that the approach can reduce the runtimes of exemplary Spark jobs on a YARN cluster by up to 12.5%, while resource utilization is increased and waiting times can be bounded." @default.
- W3129353670 created "2021-03-01" @default.
- W3129353670 creator A5014548359 @default.
- W3129353670 creator A5042349846 @default.
- W3129353670 creator A5051843551 @default.
- W3129353670 creator A5065379079 @default.
- W3129353670 creator A5065499705 @default.
- W3129353670 creator A5071130875 @default.
- W3129353670 creator A5084056435 @default.
- W3129353670 creator A5089312436 @default.
- W3129353670 date "2020-01-01" @default.
- W3129353670 modified "2023-10-02" @default.
- W3129353670 title "Hugo: A Cluster Scheduler that Efficiently Learns to Select Complementary Data-Parallel Jobs" @default.
- W3129353670 cites W2096125134 @default.
- W3129353670 cites W2098935637 @default.
- W3129353670 cites W2102458936 @default.
- W3129353670 cites W2102709380 @default.
- W3129353670 cites W2105947650 @default.
- W3129353670 cites W2129542763 @default.
- W3129353670 cites W2160121678 @default.
- W3129353670 cites W2220931077 @default.
- W3129353670 cites W2247317079 @default.
- W3129353670 cites W2276435176 @default.
- W3129353670 cites W2342245634 @default.
- W3129353670 cites W2546571074 @default.
- W3129353670 cites W2755738839 @default.
- W3129353670 cites W2962684017 @default.
- W3129353670 cites W2968986602 @default.
- W3129353670 doi "https://doi.org/10.1007/978-3-030-48340-1_40" @default.
- W3129353670 hasPublicationYear "2020" @default.
- W3129353670 type Work @default.
- W3129353670 sameAs 3129353670 @default.
- W3129353670 citedByCount "2" @default.
- W3129353670 countsByYear W31293536702020 @default.
- W3129353670 countsByYear W31293536702021 @default.
- W3129353670 crossrefType "book-chapter" @default.
- W3129353670 hasAuthorship W3129353670A5014548359 @default.
- W3129353670 hasAuthorship W3129353670A5042349846 @default.
- W3129353670 hasAuthorship W3129353670A5051843551 @default.
- W3129353670 hasAuthorship W3129353670A5065379079 @default.
- W3129353670 hasAuthorship W3129353670A5065499705 @default.
- W3129353670 hasAuthorship W3129353670A5071130875 @default.
- W3129353670 hasAuthorship W3129353670A5084056435 @default.
- W3129353670 hasAuthorship W3129353670A5089312436 @default.
- W3129353670 hasBestOaLocation W31293536702 @default.
- W3129353670 hasConcept C111873713 @default.
- W3129353670 hasConcept C111919701 @default.
- W3129353670 hasConcept C120314980 @default.
- W3129353670 hasConcept C154945302 @default.
- W3129353670 hasConcept C157764524 @default.
- W3129353670 hasConcept C159985019 @default.
- W3129353670 hasConcept C162324750 @default.
- W3129353670 hasConcept C164866538 @default.
- W3129353670 hasConcept C192562407 @default.
- W3129353670 hasConcept C199360897 @default.
- W3129353670 hasConcept C206345919 @default.
- W3129353670 hasConcept C206729178 @default.
- W3129353670 hasConcept C21547014 @default.
- W3129353670 hasConcept C2778787235 @default.
- W3129353670 hasConcept C2781215313 @default.
- W3129353670 hasConcept C29140674 @default.
- W3129353670 hasConcept C2984822820 @default.
- W3129353670 hasConcept C31258907 @default.
- W3129353670 hasConcept C41008148 @default.
- W3129353670 hasConcept C555944384 @default.
- W3129353670 hasConcept C77088390 @default.
- W3129353670 hasConcept C79974875 @default.
- W3129353670 hasConcept C97541855 @default.
- W3129353670 hasConceptScore W3129353670C111873713 @default.
- W3129353670 hasConceptScore W3129353670C111919701 @default.
- W3129353670 hasConceptScore W3129353670C120314980 @default.
- W3129353670 hasConceptScore W3129353670C154945302 @default.
- W3129353670 hasConceptScore W3129353670C157764524 @default.
- W3129353670 hasConceptScore W3129353670C159985019 @default.
- W3129353670 hasConceptScore W3129353670C162324750 @default.
- W3129353670 hasConceptScore W3129353670C164866538 @default.
- W3129353670 hasConceptScore W3129353670C192562407 @default.
- W3129353670 hasConceptScore W3129353670C199360897 @default.
- W3129353670 hasConceptScore W3129353670C206345919 @default.
- W3129353670 hasConceptScore W3129353670C206729178 @default.
- W3129353670 hasConceptScore W3129353670C21547014 @default.
- W3129353670 hasConceptScore W3129353670C2778787235 @default.
- W3129353670 hasConceptScore W3129353670C2781215313 @default.
- W3129353670 hasConceptScore W3129353670C29140674 @default.
- W3129353670 hasConceptScore W3129353670C2984822820 @default.
- W3129353670 hasConceptScore W3129353670C31258907 @default.
- W3129353670 hasConceptScore W3129353670C41008148 @default.
- W3129353670 hasConceptScore W3129353670C555944384 @default.
- W3129353670 hasConceptScore W3129353670C77088390 @default.
- W3129353670 hasConceptScore W3129353670C79974875 @default.
- W3129353670 hasConceptScore W3129353670C97541855 @default.
- W3129353670 hasLocation W31293536701 @default.
- W3129353670 hasLocation W31293536702 @default.
- W3129353670 hasOpenAccess W3129353670 @default.
- W3129353670 hasPrimaryLocation W31293536701 @default.
- W3129353670 hasRelatedWork W1572108542 @default.
- W3129353670 hasRelatedWork W2134497373 @default.
- W3129353670 hasRelatedWork W2503836759 @default.