Matches in SemOpenAlex for { <https://semopenalex.org/work/W2960779502> ?p ?o ?g. }
Showing items 1 to 82 of
82
with 100 items per page.
- W2960779502 endingPage "441" @default.
- W2960779502 startingPage "430" @default.
- W2960779502 abstract "The big table equal join operation is one of the key operations of Spark for processing large-scale data. However, when Spark handles large table equal join problems, the network transmission overhead is relatively expensive and the I/O cost is high, so this paper proposes an optimized Spark large table join method. Firstly, this method proposes a Split Compressed Bloom Filter algorithm which is suitable for filtering data sets with unknown data volume. Then, the Maxdiff histogram is used to statistically analyze the data distribution of the connected data tables, and the skew data in the data set is obtained. According to the statistical results, the RDD is split, and finally the data connection is joined by a suitable join algorithm, and the sub-results are combined to obtain the final result. Experiments show that the Spark large table equal join optimization method proposed in this paper has obvious advantages in shuffle write, shuffle read and task running time compared with Spark original method." @default.
- W2960779502 created "2019-07-23" @default.
- W2960779502 creator A5044757881 @default.
- W2960779502 creator A5074494657 @default.
- W2960779502 creator A5081924251 @default.
- W2960779502 date "2019-01-01" @default.
- W2960779502 modified "2023-09-24" @default.
- W2960779502 title "Research on the Optimization of Spark Big Table Equal Join" @default.
- W2960779502 cites W2123845384 @default.
- W2960779502 cites W2130211668 @default.
- W2960779502 cites W2136986825 @default.
- W2960779502 cites W2151065878 @default.
- W2960779502 cites W2151310484 @default.
- W2960779502 cites W2778188427 @default.
- W2960779502 cites W3145238549 @default.
- W2960779502 cites W4246006899 @default.
- W2960779502 doi "https://doi.org/10.1007/978-3-030-24265-7_37" @default.
- W2960779502 hasPublicationYear "2019" @default.
- W2960779502 type Work @default.
- W2960779502 sameAs 2960779502 @default.
- W2960779502 citedByCount "0" @default.
- W2960779502 crossrefType "book-chapter" @default.
- W2960779502 hasAuthorship W2960779502A5044757881 @default.
- W2960779502 hasAuthorship W2960779502A5074494657 @default.
- W2960779502 hasAuthorship W2960779502A5081924251 @default.
- W2960779502 hasConcept C111919701 @default.
- W2960779502 hasConcept C11413529 @default.
- W2960779502 hasConcept C114614502 @default.
- W2960779502 hasConcept C115961682 @default.
- W2960779502 hasConcept C121332964 @default.
- W2960779502 hasConcept C124101348 @default.
- W2960779502 hasConcept C154945302 @default.
- W2960779502 hasConcept C199360897 @default.
- W2960779502 hasConcept C20556612 @default.
- W2960779502 hasConcept C2776124973 @default.
- W2960779502 hasConcept C2779960059 @default.
- W2960779502 hasConcept C2781215313 @default.
- W2960779502 hasConcept C33923547 @default.
- W2960779502 hasConcept C41008148 @default.
- W2960779502 hasConcept C43711488 @default.
- W2960779502 hasConcept C45235069 @default.
- W2960779502 hasConcept C53533937 @default.
- W2960779502 hasConcept C62520636 @default.
- W2960779502 hasConcept C76155785 @default.
- W2960779502 hasConceptScore W2960779502C111919701 @default.
- W2960779502 hasConceptScore W2960779502C11413529 @default.
- W2960779502 hasConceptScore W2960779502C114614502 @default.
- W2960779502 hasConceptScore W2960779502C115961682 @default.
- W2960779502 hasConceptScore W2960779502C121332964 @default.
- W2960779502 hasConceptScore W2960779502C124101348 @default.
- W2960779502 hasConceptScore W2960779502C154945302 @default.
- W2960779502 hasConceptScore W2960779502C199360897 @default.
- W2960779502 hasConceptScore W2960779502C20556612 @default.
- W2960779502 hasConceptScore W2960779502C2776124973 @default.
- W2960779502 hasConceptScore W2960779502C2779960059 @default.
- W2960779502 hasConceptScore W2960779502C2781215313 @default.
- W2960779502 hasConceptScore W2960779502C33923547 @default.
- W2960779502 hasConceptScore W2960779502C41008148 @default.
- W2960779502 hasConceptScore W2960779502C43711488 @default.
- W2960779502 hasConceptScore W2960779502C45235069 @default.
- W2960779502 hasConceptScore W2960779502C53533937 @default.
- W2960779502 hasConceptScore W2960779502C62520636 @default.
- W2960779502 hasConceptScore W2960779502C76155785 @default.
- W2960779502 hasLocation W29607795021 @default.
- W2960779502 hasOpenAccess W2960779502 @default.
- W2960779502 hasPrimaryLocation W29607795021 @default.
- W2960779502 hasRelatedWork W2189162653 @default.
- W2960779502 hasRelatedWork W2384190089 @default.
- W2960779502 hasRelatedWork W2531041707 @default.
- W2960779502 hasRelatedWork W2542460421 @default.
- W2960779502 hasRelatedWork W2616585186 @default.
- W2960779502 hasRelatedWork W2621591477 @default.
- W2960779502 hasRelatedWork W2787123258 @default.
- W2960779502 hasRelatedWork W2960779502 @default.
- W2960779502 hasRelatedWork W3013872119 @default.
- W2960779502 hasRelatedWork W4283709512 @default.
- W2960779502 isParatext "false" @default.
- W2960779502 isRetracted "false" @default.
- W2960779502 magId "2960779502" @default.
- W2960779502 workType "book-chapter" @default.