Matches in SemOpenAlex for { <https://semopenalex.org/work/W1975293930> ?p ?o ?g. }
Showing items 1 to 80 of
80
with 100 items per page.
- W1975293930 abstract "The ever growing need to process and analyze massive amounts of data from diverse sources such as telecom call data records, telescope imagery, web pages, stock markets, medical records and other domains has triggered worldwide research in data intensive computing. A key requirement here involves removing redundancy from data, as this enhances the compute efficiency for downstream data processing. These application domains have an intense need for high throughput data deduplication for huge volumes of data flowing at the rate of 1 GB/s or more. In this paper, we present the design of a novel parallel data redundancy removal algorithm. We also present a queueing theoretic analysis to optimize the throughput of our parallel algorithm on multi-core architectures. For 500M records, our parallel algorithm can perform complete deduplication in 255s, on 16 core Intel Xeon 5570 architecture. This gives a throughput of around 2M records/s. For 2048 byte records, we achieve a throughput of 0.81 GB/s. To the best of our knowledge, this is the highest throughput for data redundancy removal on such massive datasets. We also demonstrate strong and weak scalability of our algorithm for both multi-core Power6 and Intel Xeon 5570 architectures." @default.
- W1975293930 created "2016-06-24" @default.
- W1975293930 creator A5021010333 @default.
- W1975293930 creator A5058434228 @default.
- W1975293930 creator A5065774663 @default.
- W1975293930 date "2011-01-24" @default.
- W1975293930 modified "2023-09-27" @default.
- W1975293930 title "High throughput data redundancy removal algorithm with scalable performance" @default.
- W1975293930 cites W1544599463 @default.
- W1975293930 cites W1558092379 @default.
- W1975293930 cites W1564712904 @default.
- W1975293930 cites W1993284846 @default.
- W1975293930 cites W2007842132 @default.
- W1975293930 cites W2023797161 @default.
- W1975293930 cites W2063544484 @default.
- W1975293930 cites W2108084305 @default.
- W1975293930 cites W2116967744 @default.
- W1975293930 cites W2123845384 @default.
- W1975293930 cites W2125763460 @default.
- W1975293930 cites W2125770033 @default.
- W1975293930 cites W2126540423 @default.
- W1975293930 cites W2135955253 @default.
- W1975293930 cites W2136986825 @default.
- W1975293930 cites W2142808699 @default.
- W1975293930 cites W2151823597 @default.
- W1975293930 cites W2164855881 @default.
- W1975293930 cites W2665103848 @default.
- W1975293930 doi "https://doi.org/10.1145/1944862.1944877" @default.
- W1975293930 hasPublicationYear "2011" @default.
- W1975293930 type Work @default.
- W1975293930 sameAs 1975293930 @default.
- W1975293930 citedByCount "8" @default.
- W1975293930 countsByYear W19752939302012 @default.
- W1975293930 countsByYear W19752939302013 @default.
- W1975293930 countsByYear W19752939302021 @default.
- W1975293930 crossrefType "proceedings-article" @default.
- W1975293930 hasAuthorship W1975293930A5021010333 @default.
- W1975293930 hasAuthorship W1975293930A5058434228 @default.
- W1975293930 hasAuthorship W1975293930A5065774663 @default.
- W1975293930 hasConcept C111919701 @default.
- W1975293930 hasConcept C11413529 @default.
- W1975293930 hasConcept C120314980 @default.
- W1975293930 hasConcept C145108525 @default.
- W1975293930 hasConcept C152124472 @default.
- W1975293930 hasConcept C157764524 @default.
- W1975293930 hasConcept C173608175 @default.
- W1975293930 hasConcept C32587265 @default.
- W1975293930 hasConcept C41008148 @default.
- W1975293930 hasConcept C48044578 @default.
- W1975293930 hasConcept C555944384 @default.
- W1975293930 hasConcept C77088390 @default.
- W1975293930 hasConceptScore W1975293930C111919701 @default.
- W1975293930 hasConceptScore W1975293930C11413529 @default.
- W1975293930 hasConceptScore W1975293930C120314980 @default.
- W1975293930 hasConceptScore W1975293930C145108525 @default.
- W1975293930 hasConceptScore W1975293930C152124472 @default.
- W1975293930 hasConceptScore W1975293930C157764524 @default.
- W1975293930 hasConceptScore W1975293930C173608175 @default.
- W1975293930 hasConceptScore W1975293930C32587265 @default.
- W1975293930 hasConceptScore W1975293930C41008148 @default.
- W1975293930 hasConceptScore W1975293930C48044578 @default.
- W1975293930 hasConceptScore W1975293930C555944384 @default.
- W1975293930 hasConceptScore W1975293930C77088390 @default.
- W1975293930 hasLocation W19752939301 @default.
- W1975293930 hasOpenAccess W1975293930 @default.
- W1975293930 hasPrimaryLocation W19752939301 @default.
- W1975293930 hasRelatedWork W1596201972 @default.
- W1975293930 hasRelatedWork W1967954938 @default.
- W1975293930 hasRelatedWork W1986253068 @default.
- W1975293930 hasRelatedWork W2047588290 @default.
- W1975293930 hasRelatedWork W2364921833 @default.
- W1975293930 hasRelatedWork W2380023786 @default.
- W1975293930 hasRelatedWork W2385146268 @default.
- W1975293930 hasRelatedWork W2477853911 @default.
- W1975293930 hasRelatedWork W2546696010 @default.
- W1975293930 hasRelatedWork W2503642292 @default.
- W1975293930 isParatext "false" @default.
- W1975293930 isRetracted "false" @default.
- W1975293930 magId "1975293930" @default.
- W1975293930 workType "article" @default.