Matches in SemOpenAlex for { <https://semopenalex.org/work/W3037672257> ?p ?o ?g. }
Showing items 1 to 75 of
75
with 100 items per page.
- W3037672257 endingPage "347" @default.
- W3037672257 startingPage "335" @default.
- W3037672257 abstract "Many scientific computing applications generate streams where message sizes exceed one megabyte, in contrast with smaller message sizes in enterprise contexts (order kilobytes, often XML or JSON). Furthermore, the processing cost of messages in scientific computing applications are usually an order of magnitude higher than in typical enterprise applications. Frameworks such as Apache Spark offer high throughput processing of streams with such ‘enterprise’ characteristics, as well as scalability, with high resilience and many other desirable features. Motivated by the development of near real-time image processing pipelines for roboticized microscopy, we evaluate the suitability of Apache Spark for streams more typical of scientific computing applications, those with large message sizes (up to 10 MB), and heavy per-message CPU load, under typical stream integrations. For comparison, we benchmark a P2P stream processing framework, HarmonicIO, developed in-house. Our study reveals a complex interplay of performance trade-offs, revealing the boundaries of good performance for each framework and integration over a wide domain of application loads. Based on these results, we suggest which are likely to offer good performance for a given load. Broadly, the advantages of Spark’s rich features makes its performance sensitive to message size in particular, whereas the simplicity of HarmonicIO offers more robust performance, and better CPU utilization." @default.
- W3037672257 created "2020-07-02" @default.
- W3037672257 creator A5040423646 @default.
- W3037672257 creator A5049765702 @default.
- W3037672257 creator A5065946530 @default.
- W3037672257 date "2020-01-01" @default.
- W3037672257 modified "2023-10-18" @default.
- W3037672257 title "Apache Spark Streaming, Kafka and HarmonicIO: A Performance Benchmark and Architecture Comparison for Enterprise and Scientific Computing" @default.
- W3037672257 cites W2013131338 @default.
- W3037672257 cites W2419660933 @default.
- W3037672257 cites W2486439025 @default.
- W3037672257 cites W2498111289 @default.
- W3037672257 cites W2542459869 @default.
- W3037672257 cites W2768694333 @default.
- W3037672257 cites W2811296513 @default.
- W3037672257 cites W2891018523 @default.
- W3037672257 cites W2963045412 @default.
- W3037672257 doi "https://doi.org/10.1007/978-3-030-49556-5_30" @default.
- W3037672257 hasPublicationYear "2020" @default.
- W3037672257 type Work @default.
- W3037672257 sameAs 3037672257 @default.
- W3037672257 citedByCount "7" @default.
- W3037672257 countsByYear W30376722572019 @default.
- W3037672257 countsByYear W30376722572020 @default.
- W3037672257 countsByYear W30376722572021 @default.
- W3037672257 countsByYear W30376722572023 @default.
- W3037672257 crossrefType "book-chapter" @default.
- W3037672257 hasAuthorship W3037672257A5040423646 @default.
- W3037672257 hasAuthorship W3037672257A5049765702 @default.
- W3037672257 hasAuthorship W3037672257A5065946530 @default.
- W3037672257 hasBestOaLocation W30376722572 @default.
- W3037672257 hasConcept C107027933 @default.
- W3037672257 hasConcept C111919701 @default.
- W3037672257 hasConcept C120314980 @default.
- W3037672257 hasConcept C13280743 @default.
- W3037672257 hasConcept C157764524 @default.
- W3037672257 hasConcept C185798385 @default.
- W3037672257 hasConcept C199360897 @default.
- W3037672257 hasConcept C205649164 @default.
- W3037672257 hasConcept C2781215313 @default.
- W3037672257 hasConcept C41008148 @default.
- W3037672257 hasConcept C48044578 @default.
- W3037672257 hasConcept C555944384 @default.
- W3037672257 hasConceptScore W3037672257C107027933 @default.
- W3037672257 hasConceptScore W3037672257C111919701 @default.
- W3037672257 hasConceptScore W3037672257C120314980 @default.
- W3037672257 hasConceptScore W3037672257C13280743 @default.
- W3037672257 hasConceptScore W3037672257C157764524 @default.
- W3037672257 hasConceptScore W3037672257C185798385 @default.
- W3037672257 hasConceptScore W3037672257C199360897 @default.
- W3037672257 hasConceptScore W3037672257C205649164 @default.
- W3037672257 hasConceptScore W3037672257C2781215313 @default.
- W3037672257 hasConceptScore W3037672257C41008148 @default.
- W3037672257 hasConceptScore W3037672257C48044578 @default.
- W3037672257 hasConceptScore W3037672257C555944384 @default.
- W3037672257 hasLocation W30376722571 @default.
- W3037672257 hasLocation W30376722572 @default.
- W3037672257 hasOpenAccess W3037672257 @default.
- W3037672257 hasPrimaryLocation W30376722571 @default.
- W3037672257 hasRelatedWork W2081014471 @default.
- W3037672257 hasRelatedWork W2378211422 @default.
- W3037672257 hasRelatedWork W2407107767 @default.
- W3037672257 hasRelatedWork W2564266326 @default.
- W3037672257 hasRelatedWork W2767178487 @default.
- W3037672257 hasRelatedWork W2792181990 @default.
- W3037672257 hasRelatedWork W2901787049 @default.
- W3037672257 hasRelatedWork W3044060397 @default.
- W3037672257 hasRelatedWork W3175549310 @default.
- W3037672257 hasRelatedWork W4308507533 @default.
- W3037672257 isParatext "false" @default.
- W3037672257 isRetracted "false" @default.
- W3037672257 magId "3037672257" @default.
- W3037672257 workType "book-chapter" @default.