Matches in SemOpenAlex for { <https://semopenalex.org/work/W2061037350> ?p ?o ?g. }
- W2061037350 endingPage "30" @default.
- W2061037350 startingPage "16" @default.
- W2061037350 abstract "Scientific problems that depend on processing large amounts of data require overcoming challenges in multiple areas: managing large-scale data distribution, controlling co-placement and scheduling of data with compute resources, and storing, transferring, and managing large volumes of data. Although there exist multiple approaches to address each of these challenges and the complexity of distributed environments, an integrative approach is missing; furthermore, extending existing functionality or enabling interoperable capabilities remains difficult at best. We propose the concept of Pilot-Data to address the fundamental challenges of co-placement and scheduling of data and compute in heterogeneous and distributed environments with interoperability and extensibility as first-order concerns. Pilot-Data is an extension of the Pilot-Job abstraction for supporting the management of data in conjunction with compute tasks. Pilot-Data separates logical data units from physical storage, thereby providing the basis for efficient compute/data placement and scheduling. In this paper, we discuss the design and implementation of the Pilot-Data prototype, demonstrate its use by data-intensive applications on multiple production distributed cyberinfrastructure and illustrate the advantages arising from flexible execution modes enabled by Pilot-Data. Our experiments utilize an implementation of Pilot-Data in conjunction with a scalable Pilot-Job (BigJob) to establish the application performance that can be enabled by the use of Pilot-Data. We demonstrate how the concept of Pilot-Data also provides the basis upon which to build tools and support capabilities like affinity which in turn can be used for advanced data–compute co-placement and scheduling." @default.
- W2061037350 created "2016-06-24" @default.
- W2061037350 creator A5038763846 @default.
- W2061037350 creator A5050127971 @default.
- W2061037350 creator A5071974428 @default.
- W2061037350 creator A5077700715 @default.
- W2061037350 date "2015-05-01" @default.
- W2061037350 modified "2023-10-03" @default.
- W2061037350 title "Pilot-Data: An abstraction for distributed data" @default.
- W2061037350 cites W1964941055 @default.
- W2061037350 cites W1980887185 @default.
- W2061037350 cites W1986518649 @default.
- W2061037350 cites W1998628172 @default.
- W2061037350 cites W2001569811 @default.
- W2061037350 cites W2008206224 @default.
- W2061037350 cites W2024124147 @default.
- W2061037350 cites W2034837534 @default.
- W2061037350 cites W2035362408 @default.
- W2061037350 cites W2040969944 @default.
- W2061037350 cites W2042854825 @default.
- W2061037350 cites W2057298723 @default.
- W2061037350 cites W2058479100 @default.
- W2061037350 cites W2062463655 @default.
- W2061037350 cites W2076898133 @default.
- W2061037350 cites W2077579791 @default.
- W2061037350 cites W2088379928 @default.
- W2061037350 cites W2088734805 @default.
- W2061037350 cites W2095053635 @default.
- W2061037350 cites W2099340616 @default.
- W2061037350 cites W2106494836 @default.
- W2061037350 cites W2108687825 @default.
- W2061037350 cites W2124088880 @default.
- W2061037350 cites W2166459269 @default.
- W2061037350 cites W3196112465 @default.
- W2061037350 cites W4247053599 @default.
- W2061037350 doi "https://doi.org/10.1016/j.jpdc.2014.09.009" @default.
- W2061037350 hasPublicationYear "2015" @default.
- W2061037350 type Work @default.
- W2061037350 sameAs 2061037350 @default.
- W2061037350 citedByCount "9" @default.
- W2061037350 countsByYear W20610373502014 @default.
- W2061037350 countsByYear W20610373502015 @default.
- W2061037350 countsByYear W20610373502016 @default.
- W2061037350 countsByYear W20610373502017 @default.
- W2061037350 countsByYear W20610373502018 @default.
- W2061037350 countsByYear W20610373502020 @default.
- W2061037350 countsByYear W20610373502023 @default.
- W2061037350 crossrefType "journal-article" @default.
- W2061037350 hasAuthorship W2061037350A5038763846 @default.
- W2061037350 hasAuthorship W2061037350A5050127971 @default.
- W2061037350 hasAuthorship W2061037350A5071974428 @default.
- W2061037350 hasAuthorship W2061037350A5077700715 @default.
- W2061037350 hasBestOaLocation W20610373501 @default.
- W2061037350 hasConcept C111472728 @default.
- W2061037350 hasConcept C111919701 @default.
- W2061037350 hasConcept C120314980 @default.
- W2061037350 hasConcept C124304363 @default.
- W2061037350 hasConcept C138885662 @default.
- W2061037350 hasConcept C162324750 @default.
- W2061037350 hasConcept C1668388 @default.
- W2061037350 hasConcept C199360897 @default.
- W2061037350 hasConcept C20136886 @default.
- W2061037350 hasConcept C206729178 @default.
- W2061037350 hasConcept C21547014 @default.
- W2061037350 hasConcept C2776397876 @default.
- W2061037350 hasConcept C32833848 @default.
- W2061037350 hasConcept C41008148 @default.
- W2061037350 hasConcept C48044578 @default.
- W2061037350 hasConcept C77088390 @default.
- W2061037350 hasConceptScore W2061037350C111472728 @default.
- W2061037350 hasConceptScore W2061037350C111919701 @default.
- W2061037350 hasConceptScore W2061037350C120314980 @default.
- W2061037350 hasConceptScore W2061037350C124304363 @default.
- W2061037350 hasConceptScore W2061037350C138885662 @default.
- W2061037350 hasConceptScore W2061037350C162324750 @default.
- W2061037350 hasConceptScore W2061037350C1668388 @default.
- W2061037350 hasConceptScore W2061037350C199360897 @default.
- W2061037350 hasConceptScore W2061037350C20136886 @default.
- W2061037350 hasConceptScore W2061037350C206729178 @default.
- W2061037350 hasConceptScore W2061037350C21547014 @default.
- W2061037350 hasConceptScore W2061037350C2776397876 @default.
- W2061037350 hasConceptScore W2061037350C32833848 @default.
- W2061037350 hasConceptScore W2061037350C41008148 @default.
- W2061037350 hasConceptScore W2061037350C48044578 @default.
- W2061037350 hasConceptScore W2061037350C77088390 @default.
- W2061037350 hasFunder F4320306076 @default.
- W2061037350 hasFunder F4320306084 @default.
- W2061037350 hasLocation W20610373501 @default.
- W2061037350 hasLocation W20610373502 @default.
- W2061037350 hasLocation W20610373503 @default.
- W2061037350 hasOpenAccess W2061037350 @default.
- W2061037350 hasPrimaryLocation W20610373501 @default.
- W2061037350 hasRelatedWork W1882733036 @default.
- W2061037350 hasRelatedWork W1992741870 @default.
- W2061037350 hasRelatedWork W2117848690 @default.
- W2061037350 hasRelatedWork W2159682405 @default.
- W2061037350 hasRelatedWork W2160425906 @default.
- W2061037350 hasRelatedWork W2364921833 @default.