Matches in SemOpenAlex for { <https://semopenalex.org/work/W4287121355> ?p ?o ?g. }
Showing items 1 to 54 of
54
with 100 items per page.
- W4287121355 abstract "Graph neural networks (GNNs) are powerful tools for learning from graph data and are widely used in various applications such as social network recommendation, fraud detection, and graph search. The graphs in these applications are typically large, usually containing hundreds of millions of nodes. Training GNN models on such large graphs efficiently remains a big challenge. Despite a number of sampling-based methods have been proposed to enable mini-batch training on large graphs, these methods have not been proved to work on truly industry-scale graphs, which require GPUs or mixed-CPU-GPU training. The state-of-the-art sampling-based methods are usually not optimized for these real-world hardware setups, in which data movement between CPUs and GPUs is a bottleneck. To address this issue, we propose Global Neighborhood Sampling that aims at training GNNs on giant graphs specifically for mixed-CPU-GPU training. The algorithm samples a global cache of nodes periodically for all mini-batches and stores them in GPUs. This global cache allows in-GPU importance sampling of mini-batches, which drastically reduces the number of nodes in a mini-batch, especially in the input layer, to reduce data copy between CPU and GPU and mini-batch computation without compromising the training convergence rate or model accuracy. We provide a highly efficient implementation of this method and show that our implementation outperforms an efficient node-wise neighbor sampling baseline by a factor of 2X-4X on giant graphs. It outperforms an efficient implementation of LADIES with small layers by a factor of 2X-14X while achieving much higher accuracy than LADIES.We also theoretically analyze the proposed algorithm and show that with cached node data of a proper size, it enjoys a comparable convergence rate as the underlying node-wise sampling method." @default.
- W4287121355 created "2022-07-25" @default.
- W4287121355 creator A5004494814 @default.
- W4287121355 creator A5005859555 @default.
- W4287121355 creator A5062596290 @default.
- W4287121355 creator A5072096775 @default.
- W4287121355 date "2021-06-10" @default.
- W4287121355 modified "2023-10-18" @default.
- W4287121355 title "Global Neighbor Sampling for Mixed CPU-GPU Training on Giant Graphs" @default.
- W4287121355 hasPublicationYear "2021" @default.
- W4287121355 type Work @default.
- W4287121355 citedByCount "0" @default.
- W4287121355 crossrefType "posted-content" @default.
- W4287121355 hasAuthorship W4287121355A5004494814 @default.
- W4287121355 hasAuthorship W4287121355A5005859555 @default.
- W4287121355 hasAuthorship W4287121355A5062596290 @default.
- W4287121355 hasAuthorship W4287121355A5072096775 @default.
- W4287121355 hasBestOaLocation W42871213551 @default.
- W4287121355 hasConcept C106131492 @default.
- W4287121355 hasConcept C115537543 @default.
- W4287121355 hasConcept C132525143 @default.
- W4287121355 hasConcept C140779682 @default.
- W4287121355 hasConcept C149635348 @default.
- W4287121355 hasConcept C173608175 @default.
- W4287121355 hasConcept C2780513914 @default.
- W4287121355 hasConcept C31972630 @default.
- W4287121355 hasConcept C41008148 @default.
- W4287121355 hasConcept C80444323 @default.
- W4287121355 hasConceptScore W4287121355C106131492 @default.
- W4287121355 hasConceptScore W4287121355C115537543 @default.
- W4287121355 hasConceptScore W4287121355C132525143 @default.
- W4287121355 hasConceptScore W4287121355C140779682 @default.
- W4287121355 hasConceptScore W4287121355C149635348 @default.
- W4287121355 hasConceptScore W4287121355C173608175 @default.
- W4287121355 hasConceptScore W4287121355C2780513914 @default.
- W4287121355 hasConceptScore W4287121355C31972630 @default.
- W4287121355 hasConceptScore W4287121355C41008148 @default.
- W4287121355 hasConceptScore W4287121355C80444323 @default.
- W4287121355 hasLocation W42871213551 @default.
- W4287121355 hasOpenAccess W4287121355 @default.
- W4287121355 hasPrimaryLocation W42871213551 @default.
- W4287121355 hasRelatedWork W1353218 @default.
- W4287121355 hasRelatedWork W13701174 @default.
- W4287121355 hasRelatedWork W137529 @default.
- W4287121355 hasRelatedWork W14360457 @default.
- W4287121355 hasRelatedWork W3528745 @default.
- W4287121355 hasRelatedWork W5231949 @default.
- W4287121355 hasRelatedWork W5243928 @default.
- W4287121355 hasRelatedWork W6061459 @default.
- W4287121355 hasRelatedWork W6348178 @default.
- W4287121355 hasRelatedWork W7880675 @default.
- W4287121355 isParatext "false" @default.
- W4287121355 isRetracted "false" @default.
- W4287121355 workType "article" @default.