Matches in SemOpenAlex for { <https://semopenalex.org/work/W4308393414> ?p ?o ?g. }
Showing items 1 to 63 of
63
with 100 items per page.
- W4308393414 abstract "Knowledge distillation is a learning paradigm for boosting resource-efficient graph neural networks (GNNs) using more expressive yet cumbersome teacher models. Past work on distillation for GNNs proposed the Local Structure Preserving loss (LSP), which matches local structural relationships defined over edges across the student and teacher's node embeddings. This paper studies whether preserving the global topology of how the teacher embeds graph data can be a more effective distillation objective for GNNs, as real-world graphs often contain latent interactions and noisy edges. We propose Graph Contrastive Representation Distillation (G-CRD), which uses contrastive learning to implicitly preserve global topology by aligning the student node embeddings to those of the teacher in a shared representation space. Additionally, we introduce an expanded set of benchmarks on large-scale real-world datasets where the performance gap between teacher and student GNNs is non-negligible. Experiments across 4 datasets and 14 heterogeneous GNN architectures show that G-CRD consistently boosts the performance and robustness of lightweight GNNs, outperforming LSP (and a global structure preserving variant of LSP) as well as baselines from 2D computer vision. An analysis of the representational similarity among teacher and student embedding spaces reveals that G-CRD balances preserving local and global relationships, while structure preserving approaches are best at preserving one or the other." @default.
- W4308393414 created "2022-11-11" @default.
- W4308393414 creator A5001103949 @default.
- W4308393414 creator A5036457690 @default.
- W4308393414 creator A5041429006 @default.
- W4308393414 creator A5042271198 @default.
- W4308393414 creator A5084138424 @default.
- W4308393414 date "2021-11-09" @default.
- W4308393414 modified "2023-09-23" @default.
- W4308393414 title "On Representation Knowledge Distillation for Graph Neural Networks" @default.
- W4308393414 doi "https://doi.org/10.48550/arxiv.2111.04964" @default.
- W4308393414 hasPublicationYear "2021" @default.
- W4308393414 type Work @default.
- W4308393414 citedByCount "0" @default.
- W4308393414 crossrefType "posted-content" @default.
- W4308393414 hasAuthorship W4308393414A5001103949 @default.
- W4308393414 hasAuthorship W4308393414A5036457690 @default.
- W4308393414 hasAuthorship W4308393414A5041429006 @default.
- W4308393414 hasAuthorship W4308393414A5042271198 @default.
- W4308393414 hasAuthorship W4308393414A5084138424 @default.
- W4308393414 hasBestOaLocation W43083934141 @default.
- W4308393414 hasConcept C104317684 @default.
- W4308393414 hasConcept C119857082 @default.
- W4308393414 hasConcept C132525143 @default.
- W4308393414 hasConcept C154945302 @default.
- W4308393414 hasConcept C185592680 @default.
- W4308393414 hasConcept C41008148 @default.
- W4308393414 hasConcept C41608201 @default.
- W4308393414 hasConcept C46686674 @default.
- W4308393414 hasConcept C50644808 @default.
- W4308393414 hasConcept C55493867 @default.
- W4308393414 hasConcept C59404180 @default.
- W4308393414 hasConcept C63479239 @default.
- W4308393414 hasConcept C80444323 @default.
- W4308393414 hasConceptScore W4308393414C104317684 @default.
- W4308393414 hasConceptScore W4308393414C119857082 @default.
- W4308393414 hasConceptScore W4308393414C132525143 @default.
- W4308393414 hasConceptScore W4308393414C154945302 @default.
- W4308393414 hasConceptScore W4308393414C185592680 @default.
- W4308393414 hasConceptScore W4308393414C41008148 @default.
- W4308393414 hasConceptScore W4308393414C41608201 @default.
- W4308393414 hasConceptScore W4308393414C46686674 @default.
- W4308393414 hasConceptScore W4308393414C50644808 @default.
- W4308393414 hasConceptScore W4308393414C55493867 @default.
- W4308393414 hasConceptScore W4308393414C59404180 @default.
- W4308393414 hasConceptScore W4308393414C63479239 @default.
- W4308393414 hasConceptScore W4308393414C80444323 @default.
- W4308393414 hasLocation W43083934141 @default.
- W4308393414 hasOpenAccess W4308393414 @default.
- W4308393414 hasPrimaryLocation W43083934141 @default.
- W4308393414 hasRelatedWork W2980233312 @default.
- W4308393414 hasRelatedWork W2990028173 @default.
- W4308393414 hasRelatedWork W3035116611 @default.
- W4308393414 hasRelatedWork W3036413464 @default.
- W4308393414 hasRelatedWork W3094605108 @default.
- W4308393414 hasRelatedWork W3129758539 @default.
- W4308393414 hasRelatedWork W3149439221 @default.
- W4308393414 hasRelatedWork W3208308319 @default.
- W4308393414 hasRelatedWork W4287755603 @default.
- W4308393414 hasRelatedWork W4287763734 @default.
- W4308393414 isParatext "false" @default.
- W4308393414 isRetracted "false" @default.
- W4308393414 workType "article" @default.