Matches in SemOpenAlex for { <https://semopenalex.org/work/W4313305625> ?p ?o ?g. }
Showing items 1 to 65 of
65
with 100 items per page.
- W4313305625 abstract "The paradigm of big data is characterized by the need to collect and process data sets of great volume, arriving at the systems with great velocity, in a variety of formats. Spark is a widely used big data processing system that can be integrated with Hadoop to provide powerful abstractions to developers, such as distributed storage through HDFS and resource management through YARN. When all the required configurations are made, Spark can also provide quality attributes, such as scalability, fault tolerance, and security. However, all of these benefits come at the cost of complexity, with high memory requirements, and additional latency in processing. An alternative approach is to use a lean software stack, like Unicage, that delegates most control back to the developer. In this work we evaluated the performance of big data processing with Spark versus Unicage, in a cluster environment hosted in the IBM Cloud. Two sets of experiments were performed: batch processing of unstructured data sets, and query processing of structured data sets. The input data sets were of significant size, ranging from 64 GB to 8192 GB in volume. The results show that the performance of Unicage scripts is superior to Spark for search workloads like grep and select, but that the abstractions of distributed storage and resource management from the Hadoop stack enable Spark to execute workloads with inter-record dependencies, such as sort and join, with correct outputs." @default.
- W4313305625 created "2023-01-06" @default.
- W4313305625 creator A5026431760 @default.
- W4313305625 creator A5060257664 @default.
- W4313305625 creator A5088970281 @default.
- W4313305625 date "2022-12-27" @default.
- W4313305625 modified "2023-09-23" @default.
- W4313305625 title "Does Big Data Require Complex Systems? A Performance Comparison Between Spark and Unicage Shell Scripts" @default.
- W4313305625 doi "https://doi.org/10.48550/arxiv.2212.13647" @default.
- W4313305625 hasPublicationYear "2022" @default.
- W4313305625 type Work @default.
- W4313305625 citedByCount "0" @default.
- W4313305625 crossrefType "posted-content" @default.
- W4313305625 hasAuthorship W4313305625A5026431760 @default.
- W4313305625 hasAuthorship W4313305625A5060257664 @default.
- W4313305625 hasAuthorship W4313305625A5088970281 @default.
- W4313305625 hasBestOaLocation W43133056251 @default.
- W4313305625 hasConcept C111919701 @default.
- W4313305625 hasConcept C120314980 @default.
- W4313305625 hasConcept C121332964 @default.
- W4313305625 hasConcept C171250308 @default.
- W4313305625 hasConcept C192562407 @default.
- W4313305625 hasConcept C199360897 @default.
- W4313305625 hasConcept C20556612 @default.
- W4313305625 hasConcept C2780940931 @default.
- W4313305625 hasConcept C2781215313 @default.
- W4313305625 hasConcept C41008148 @default.
- W4313305625 hasConcept C48044578 @default.
- W4313305625 hasConcept C61423126 @default.
- W4313305625 hasConcept C62520636 @default.
- W4313305625 hasConcept C70388272 @default.
- W4313305625 hasConcept C75684735 @default.
- W4313305625 hasConcept C77088390 @default.
- W4313305625 hasConceptScore W4313305625C111919701 @default.
- W4313305625 hasConceptScore W4313305625C120314980 @default.
- W4313305625 hasConceptScore W4313305625C121332964 @default.
- W4313305625 hasConceptScore W4313305625C171250308 @default.
- W4313305625 hasConceptScore W4313305625C192562407 @default.
- W4313305625 hasConceptScore W4313305625C199360897 @default.
- W4313305625 hasConceptScore W4313305625C20556612 @default.
- W4313305625 hasConceptScore W4313305625C2780940931 @default.
- W4313305625 hasConceptScore W4313305625C2781215313 @default.
- W4313305625 hasConceptScore W4313305625C41008148 @default.
- W4313305625 hasConceptScore W4313305625C48044578 @default.
- W4313305625 hasConceptScore W4313305625C61423126 @default.
- W4313305625 hasConceptScore W4313305625C62520636 @default.
- W4313305625 hasConceptScore W4313305625C70388272 @default.
- W4313305625 hasConceptScore W4313305625C75684735 @default.
- W4313305625 hasConceptScore W4313305625C77088390 @default.
- W4313305625 hasLocation W43133056251 @default.
- W4313305625 hasOpenAccess W4313305625 @default.
- W4313305625 hasPrimaryLocation W43133056251 @default.
- W4313305625 hasRelatedWork W2189081352 @default.
- W4313305625 hasRelatedWork W2364921833 @default.
- W4313305625 hasRelatedWork W2553660239 @default.
- W4313305625 hasRelatedWork W2782700877 @default.
- W4313305625 hasRelatedWork W280853923 @default.
- W4313305625 hasRelatedWork W2889616422 @default.
- W4313305625 hasRelatedWork W2891888092 @default.
- W4313305625 hasRelatedWork W2900588685 @default.
- W4313305625 hasRelatedWork W2970080579 @default.
- W4313305625 hasRelatedWork W3185293612 @default.
- W4313305625 isParatext "false" @default.
- W4313305625 isRetracted "false" @default.
- W4313305625 workType "article" @default.