Matches in SemOpenAlex for { <https://semopenalex.org/work/W3137018880> ?p ?o ?g. }
Showing items 1 to 66 of
66
with 100 items per page.
- W3137018880 abstract "Data skipping reduces I/O for SQL queries by skipping over irrelevant data objects (files) based on their metadata. We extend this notion by allowing developers to define their own data s kipping metadata types and indexes using a flexible A PI. Our framework i s t he first to natively support data skipping for arbitrary data types (e.g. geospatial, logs) and queries with User Defined Functions ( UDFs). We integrated our framework with Apache Spark and it is now deployed across multiple products/services at IBM. We present our extensible data skipping APIs, discuss index design, and implement various metadata indexes, requiring only around 30 lines of additional code per index. In particular we implement data skipping for a third party library with geospatial UDFs and demonstrate speedups of two orders of magnitude. Our centralized metadata approach provides a x3.6 speed up even when compared to queries which are rewritten to exploit Parquet min/max metadata. We demonstrate that extensible data skipping is applicable to broad class of applications, where user defined indexes achieve significant speedups and cost savings with very low development cost." @default.
- W3137018880 created "2021-03-29" @default.
- W3137018880 creator A5007261068 @default.
- W3137018880 creator A5016766945 @default.
- W3137018880 creator A5075862099 @default.
- W3137018880 creator A5080554083 @default.
- W3137018880 date "2020-12-10" @default.
- W3137018880 modified "2023-09-25" @default.
- W3137018880 title "Extensible Data Skipping" @default.
- W3137018880 cites W1967601791 @default.
- W3137018880 cites W1981988185 @default.
- W3137018880 cites W2028226582 @default.
- W3137018880 cites W2038412523 @default.
- W3137018880 cites W2106867122 @default.
- W3137018880 cites W2109966155 @default.
- W3137018880 cites W2123845384 @default.
- W3137018880 cites W2168456130 @default.
- W3137018880 cites W2280230190 @default.
- W3137018880 cites W2591324491 @default.
- W3137018880 cites W2612529667 @default.
- W3137018880 cites W2750787376 @default.
- W3137018880 cites W2756982556 @default.
- W3137018880 cites W2798891709 @default.
- W3137018880 cites W2799221749 @default.
- W3137018880 cites W2805382875 @default.
- W3137018880 cites W2889201433 @default.
- W3137018880 cites W2992496917 @default.
- W3137018880 cites W3137018880 @default.
- W3137018880 doi "https://doi.org/10.1109/bigdata50022.2020.9377740" @default.
- W3137018880 hasPublicationYear "2020" @default.
- W3137018880 type Work @default.
- W3137018880 sameAs 3137018880 @default.
- W3137018880 citedByCount "3" @default.
- W3137018880 countsByYear W31370188802020 @default.
- W3137018880 countsByYear W31370188802021 @default.
- W3137018880 countsByYear W31370188802022 @default.
- W3137018880 crossrefType "proceedings-article" @default.
- W3137018880 hasAuthorship W3137018880A5007261068 @default.
- W3137018880 hasAuthorship W3137018880A5016766945 @default.
- W3137018880 hasAuthorship W3137018880A5075862099 @default.
- W3137018880 hasAuthorship W3137018880A5080554083 @default.
- W3137018880 hasBestOaLocation W31370188802 @default.
- W3137018880 hasConcept C199360897 @default.
- W3137018880 hasConcept C32833848 @default.
- W3137018880 hasConcept C41008148 @default.
- W3137018880 hasConceptScore W3137018880C199360897 @default.
- W3137018880 hasConceptScore W3137018880C32833848 @default.
- W3137018880 hasConceptScore W3137018880C41008148 @default.
- W3137018880 hasLocation W31370188801 @default.
- W3137018880 hasLocation W31370188802 @default.
- W3137018880 hasOpenAccess W3137018880 @default.
- W3137018880 hasPrimaryLocation W31370188801 @default.
- W3137018880 hasRelatedWork W1984116007 @default.
- W3137018880 hasRelatedWork W2051744418 @default.
- W3137018880 hasRelatedWork W2165124476 @default.
- W3137018880 hasRelatedWork W2362760518 @default.
- W3137018880 hasRelatedWork W2364420803 @default.
- W3137018880 hasRelatedWork W2368425793 @default.
- W3137018880 hasRelatedWork W2372886617 @default.
- W3137018880 hasRelatedWork W25068511 @default.
- W3137018880 hasRelatedWork W3013297713 @default.
- W3137018880 hasRelatedWork W2090691072 @default.
- W3137018880 isParatext "false" @default.
- W3137018880 isRetracted "false" @default.
- W3137018880 magId "3137018880" @default.
- W3137018880 workType "article" @default.