FRINK Linked Data Fragments server

Matches in SemOpenAlex for { <https://semopenalex.org/work/W4226339027> ?p ?o ?g. }

• W4226339027 endingPage "948" @default.
• W4226339027 startingPage "936" @default.
• W4226339027 abstract "The cost of big-data query execution is dominated by stateful operators. These include sort and hash-aggregate that typically materialize intermediate data in memory, and exchange that materializes data to disk and transfers data over the network. In this paper we focus on several query optimization techniques that reduce the cost of these operators. First, we introduce a novel exchange placement algorithm that improves the state-of-the-art and significantly reduces the amount of data exchanged. The algorithm simultaneously minimizes the number of exchanges required and maximizes computation reuse via multi-consumer exchanges. Second, we introduce three partial push-down optimizations that push down partial computation derived from existing operators ( group-bys , intersections and joins ) below these stateful operators. While these optimizations are generically applicable we find that two of these optimizations ( partial aggregate and partial semi-join push-down ) are only beneficial in the scale-out setting where exchanges are a bottleneck. We propose novel extensions to existing literature to perform more aggressive partial push-downs than the state-of-the-art and also specialize them to the big-data setting. Finally we propose peephole optimizations that specialize the implementation of stateful operators to their input parameters. All our optimizations are implemented in the spark engine that powers azure synapse. We evaluate their impact on TPCDS and demonstrate that they make our engine 1.8X faster than Apache Spark 3.0.1." @default.
• W4226339027 created "2022-05-05" @default.
• W4226339027 creator A5001318607 @default.
• W4226339027 creator A5008493293 @default.
• W4226339027 creator A5009481416 @default.
• W4226339027 creator A5013812903 @default.
• W4226339027 creator A5023086059 @default.
• W4226339027 creator A5035678653 @default.
• W4226339027 creator A5062222915 @default.
• W4226339027 creator A5071590490 @default.
• W4226339027 creator A5080077704 @default.
• W4226339027 creator A5080471573 @default.
• W4226339027 date "2021-12-01" @default.
• W4226339027 modified "2023-09-30" @default.
• W4226339027 title "New query optimization techniques in the Spark engine of Azure synapse" @default.
• W4226339027 cites W1967373117 @default.
• W4226339027 cites W1991271936 @default.
• W4226339027 cites W1993433750 @default.
• W4226339027 cites W2038412523 @default.
• W4226339027 cites W2081869611 @default.
• W4226339027 cites W2089347259 @default.
• W4226339027 cites W2110086534 @default.
• W4226339027 cites W2110816960 @default.
• W4226339027 cites W2114103324 @default.
• W4226339027 cites W2116436709 @default.
• W4226339027 cites W2123845384 @default.
• W4226339027 cites W2124645273 @default.
• W4226339027 cites W2151251992 @default.
• W4226339027 cites W2158237121 @default.
• W4226339027 cites W2430301697 @default.
• W4226339027 cites W2798457757 @default.
• W4226339027 cites W2799237774 @default.
• W4226339027 cites W2949762319 @default.
• W4226339027 cites W2965018236 @default.
• W4226339027 cites W3028610005 @default.
• W4226339027 cites W3084687783 @default.
• W4226339027 doi "https://doi.org/10.14778/3503585.3503601" @default.
• W4226339027 hasPublicationYear "2021" @default.
• W4226339027 type Work @default.
• W4226339027 citedByCount "5" @default.
• W4226339027 countsByYear W42263390272023 @default.
• W4226339027 crossrefType "journal-article" @default.
• W4226339027 hasAuthorship W4226339027A5001318607 @default.
• W4226339027 hasAuthorship W4226339027A5008493293 @default.
• W4226339027 hasAuthorship W4226339027A5009481416 @default.
• W4226339027 hasAuthorship W4226339027A5013812903 @default.
• W4226339027 hasAuthorship W4226339027A5023086059 @default.
• W4226339027 hasAuthorship W4226339027A5035678653 @default.
• W4226339027 hasAuthorship W4226339027A5062222915 @default.
• W4226339027 hasAuthorship W4226339027A5071590490 @default.
• W4226339027 hasAuthorship W4226339027A5080077704 @default.
• W4226339027 hasAuthorship W4226339027A5080471573 @default.
• W4226339027 hasConcept C111919701 @default.
• W4226339027 hasConcept C11413529 @default.
• W4226339027 hasConcept C120314980 @default.
• W4226339027 hasConcept C149635348 @default.
• W4226339027 hasConcept C158379750 @default.
• W4226339027 hasConcept C173608175 @default.
• W4226339027 hasConcept C199360897 @default.
• W4226339027 hasConcept C22927095 @default.
• W4226339027 hasConcept C2524010 @default.
• W4226339027 hasConcept C2777210771 @default.
• W4226339027 hasConcept C2778692605 @default.
• W4226339027 hasConcept C2780513914 @default.
• W4226339027 hasConcept C2781215313 @default.
• W4226339027 hasConcept C31258907 @default.
• W4226339027 hasConcept C33923547 @default.
• W4226339027 hasConcept C41008148 @default.
• W4226339027 hasConcept C45374587 @default.
• W4226339027 hasConcept C75684735 @default.
• W4226339027 hasConcept C99138194 @default.
• W4226339027 hasConceptScore W4226339027C111919701 @default.
• W4226339027 hasConceptScore W4226339027C11413529 @default.
• W4226339027 hasConceptScore W4226339027C120314980 @default.
• W4226339027 hasConceptScore W4226339027C149635348 @default.
• W4226339027 hasConceptScore W4226339027C158379750 @default.
• W4226339027 hasConceptScore W4226339027C173608175 @default.
• W4226339027 hasConceptScore W4226339027C199360897 @default.
• W4226339027 hasConceptScore W4226339027C22927095 @default.
• W4226339027 hasConceptScore W4226339027C2524010 @default.
• W4226339027 hasConceptScore W4226339027C2777210771 @default.
• W4226339027 hasConceptScore W4226339027C2778692605 @default.
• W4226339027 hasConceptScore W4226339027C2780513914 @default.
• W4226339027 hasConceptScore W4226339027C2781215313 @default.
• W4226339027 hasConceptScore W4226339027C31258907 @default.
• W4226339027 hasConceptScore W4226339027C33923547 @default.
• W4226339027 hasConceptScore W4226339027C41008148 @default.
• W4226339027 hasConceptScore W4226339027C45374587 @default.
• W4226339027 hasConceptScore W4226339027C75684735 @default.
• W4226339027 hasConceptScore W4226339027C99138194 @default.
• W4226339027 hasIssue "4" @default.
• W4226339027 hasLocation W42263390271 @default.
• W4226339027 hasOpenAccess W4226339027 @default.
• W4226339027 hasPrimaryLocation W42263390271 @default.
• W4226339027 hasRelatedWork W1535543249 @default.
• W4226339027 hasRelatedWork W2068088848 @default.
• W4226339027 hasRelatedWork W2158013543 @default.
• W4226339027 hasRelatedWork W2362309373 @default.

• next