Matches in SemOpenAlex for { <https://semopenalex.org/work/W2903816298> ?p ?o ?g. }
Showing items 1 to 96 of
96
with 100 items per page.
- W2903816298 abstract "In the parallel computing framework of Hadoop/Spark, data skew is a common problem resulting in performance degradation, such as prolonging of the entire execution time and idle resources. What lies behind this issue is partition imbalance, which causes significant differences in the amount of data processed by each reduce task. This paper proposes a key reassigning and splitting partition algorithm (SKRSP) to handle skew, which considers both the partition balance of the intermediate data and the partition balance after shuffle operator. We design two partition algorithms for different applications: the range-based key splitting partition method (KSRP) for sort operation and hash-based key reassigning partition method (KRHP) for the other operations. We implement SKRSP in Spark 2.2.0 and evaluate its performance through three benchmarks exhibiting significant data skew: Sort, Join, and PageRank. The experimental results verify that our algorithm not only can achieve a better partition balance but also reduce the execution time of reduce tasks effectively." @default.
- W2903816298 created "2018-12-22" @default.
- W2903816298 creator A5008574161 @default.
- W2903816298 creator A5056242893 @default.
- W2903816298 creator A5078793726 @default.
- W2903816298 date "2018-10-01" @default.
- W2903816298 modified "2023-09-25" @default.
- W2903816298 title "An Adaptive Partition Method for Handling Skew in Spark Applications" @default.
- W2903816298 cites W2002722920 @default.
- W2903816298 cites W2007333650 @default.
- W2903816298 cites W2024077218 @default.
- W2903816298 cites W2037027334 @default.
- W2903816298 cites W2041958520 @default.
- W2903816298 cites W2044936152 @default.
- W2903816298 cites W2048455627 @default.
- W2903816298 cites W2060650311 @default.
- W2903816298 cites W2061807980 @default.
- W2903816298 cites W2080131844 @default.
- W2903816298 cites W2120755171 @default.
- W2903816298 cites W2121562043 @default.
- W2903816298 cites W2131975293 @default.
- W2903816298 cites W2137373960 @default.
- W2903816298 cites W2140509629 @default.
- W2903816298 cites W2165769837 @default.
- W2903816298 cites W2189465200 @default.
- W2903816298 cites W2399133307 @default.
- W2903816298 cites W2439404901 @default.
- W2903816298 cites W2464834590 @default.
- W2903816298 cites W2515208787 @default.
- W2903816298 cites W2517850251 @default.
- W2903816298 cites W2526832688 @default.
- W2903816298 cites W2734430573 @default.
- W2903816298 doi "https://doi.org/10.1109/smartworld.2018.00185" @default.
- W2903816298 hasPublicationYear "2018" @default.
- W2903816298 type Work @default.
- W2903816298 sameAs 2903816298 @default.
- W2903816298 citedByCount "2" @default.
- W2903816298 countsByYear W29038162982020 @default.
- W2903816298 countsByYear W29038162982023 @default.
- W2903816298 crossrefType "proceedings-article" @default.
- W2903816298 hasAuthorship W2903816298A5008574161 @default.
- W2903816298 hasAuthorship W2903816298A5056242893 @default.
- W2903816298 hasAuthorship W2903816298A5078793726 @default.
- W2903816298 hasConcept C111919701 @default.
- W2903816298 hasConcept C11413529 @default.
- W2903816298 hasConcept C114614502 @default.
- W2903816298 hasConcept C120314980 @default.
- W2903816298 hasConcept C173608175 @default.
- W2903816298 hasConcept C199360897 @default.
- W2903816298 hasConcept C26517878 @default.
- W2903816298 hasConcept C2781215313 @default.
- W2903816298 hasConcept C33923547 @default.
- W2903816298 hasConcept C35995877 @default.
- W2903816298 hasConcept C38652104 @default.
- W2903816298 hasConcept C41008148 @default.
- W2903816298 hasConcept C42812 @default.
- W2903816298 hasConcept C43711488 @default.
- W2903816298 hasConcept C76155785 @default.
- W2903816298 hasConcept C77088390 @default.
- W2903816298 hasConcept C88548561 @default.
- W2903816298 hasConcept C99138194 @default.
- W2903816298 hasConceptScore W2903816298C111919701 @default.
- W2903816298 hasConceptScore W2903816298C11413529 @default.
- W2903816298 hasConceptScore W2903816298C114614502 @default.
- W2903816298 hasConceptScore W2903816298C120314980 @default.
- W2903816298 hasConceptScore W2903816298C173608175 @default.
- W2903816298 hasConceptScore W2903816298C199360897 @default.
- W2903816298 hasConceptScore W2903816298C26517878 @default.
- W2903816298 hasConceptScore W2903816298C2781215313 @default.
- W2903816298 hasConceptScore W2903816298C33923547 @default.
- W2903816298 hasConceptScore W2903816298C35995877 @default.
- W2903816298 hasConceptScore W2903816298C38652104 @default.
- W2903816298 hasConceptScore W2903816298C41008148 @default.
- W2903816298 hasConceptScore W2903816298C42812 @default.
- W2903816298 hasConceptScore W2903816298C43711488 @default.
- W2903816298 hasConceptScore W2903816298C76155785 @default.
- W2903816298 hasConceptScore W2903816298C77088390 @default.
- W2903816298 hasConceptScore W2903816298C88548561 @default.
- W2903816298 hasConceptScore W2903816298C99138194 @default.
- W2903816298 hasLocation W29038162981 @default.
- W2903816298 hasOpenAccess W2903816298 @default.
- W2903816298 hasPrimaryLocation W29038162981 @default.
- W2903816298 hasRelatedWork W1495086522 @default.
- W2903816298 hasRelatedWork W1570320352 @default.
- W2903816298 hasRelatedWork W1974289416 @default.
- W2903816298 hasRelatedWork W2056043698 @default.
- W2903816298 hasRelatedWork W2363209164 @default.
- W2903816298 hasRelatedWork W2377549071 @default.
- W2903816298 hasRelatedWork W2895095618 @default.
- W2903816298 hasRelatedWork W2899303430 @default.
- W2903816298 hasRelatedWork W2903816298 @default.
- W2903816298 hasRelatedWork W3182128720 @default.
- W2903816298 isParatext "false" @default.
- W2903816298 isRetracted "false" @default.
- W2903816298 magId "2903816298" @default.
- W2903816298 workType "article" @default.