FRINK Linked Data Fragments server

Matches in SemOpenAlex for { <https://semopenalex.org/work/W3212259012> ?p ?o ?g. }

• W3212259012 abstract "We present a generalization of Transformers to any-order permutation invariant data (sets, graphs, and hypergraphs). We begin by observing that Transformers generalize DeepSets, or first-order (set-input) permutation invariant MLPs. Then, based on recently characterized higher-order invariant MLPs, we extend the concept of self-attention to higher orders and propose higher-order Transformers for order-$k$ data ($k=2$ for graphs and $k>2$ for hypergraphs). Unfortunately, higher-order Transformers turn out to have prohibitive complexity $mathcal{O}(n^{2k})$ to the number of input nodes $n$. To address this problem, we present sparse higher-order Transformers that have quadratic complexity to the number of input hyperedges, and further adopt the kernel attention approach to reduce the complexity to linear. In particular, we show that the sparse second-order Transformers with kernel attention are theoretically more expressive than message passing operations while having an asymptotically identical complexity. Our models achieve significant performance improvement over invariant MLPs and message-passing graph neural networks in large-scale graph regression and set-to-(hyper)graph prediction tasks. Our implementation is available at this https URL." @default.
• W3212259012 created "2021-11-22" @default.
• W3212259012 creator A5021505488 @default.
• W3212259012 creator A5072157215 @default.
• W3212259012 creator A5077461583 @default.
• W3212259012 date "2021-10-27" @default.
• W3212259012 modified "2023-09-27" @default.
• W3212259012 title "Transformers Generalize DeepSets and Can be Extended to Graphs and Hypergraphs." @default.
• W3212259012 cites W2095705004 @default.
• W3212259012 cites W2097308346 @default.
• W3212259012 cites W2117587045 @default.
• W3212259012 cites W2219888463 @default.
• W3212259012 cites W2335915014 @default.
• W3212259012 cites W2606780347 @default.
• W3212259012 cites W2751808960 @default.
• W3212259012 cites W2753798143 @default.
• W3212259012 cites W2766099245 @default.
• W3212259012 cites W2809583854 @default.
• W3212259012 cites W2899663614 @default.
• W3212259012 cites W2904900486 @default.
• W3212259012 cites W2905224888 @default.
• W3212259012 cites W2906340703 @default.
• W3212259012 cites W2908510526 @default.
• W3212259012 cites W2953273646 @default.
• W3212259012 cites W2963403868 @default.
• W3212259012 cites W2963464736 @default.
• W3212259012 cites W2963887288 @default.
• W3212259012 cites W2964051675 @default.
• W3212259012 cites W2964121744 @default.
• W3212259012 cites W2970493342 @default.
• W3212259012 cites W2971078096 @default.
• W3212259012 cites W2985748879 @default.
• W3212259012 cites W2995273672 @default.
• W3212259012 cites W2995914187 @default.
• W3212259012 cites W2996226706 @default.
• W3212259012 cites W2997347790 @default.
• W3212259012 cites W3034573343 @default.
• W3212259012 cites W3037242362 @default.
• W3212259012 cites W3037932933 @default.
• W3212259012 cites W3103412713 @default.
• W3212259012 cites W3104801438 @default.
• W3212259012 cites W3113177135 @default.
• W3212259012 cites W3125056032 @default.
• W3212259012 cites W3136399186 @default.
• W3212259012 cites W3146803896 @default.
• W3212259012 hasPublicationYear "2021" @default.
• W3212259012 type Work @default.
• W3212259012 sameAs 3212259012 @default.
• W3212259012 citedByCount "0" @default.
• W3212259012 crossrefType "posted-content" @default.
• W3212259012 hasAuthorship W3212259012A5021505488 @default.
• W3212259012 hasAuthorship W3212259012A5072157215 @default.
• W3212259012 hasAuthorship W3212259012A5077461583 @default.
• W3212259012 hasConcept C11413529 @default.
• W3212259012 hasConcept C114614502 @default.
• W3212259012 hasConcept C118615104 @default.
• W3212259012 hasConcept C121332964 @default.
• W3212259012 hasConcept C129844170 @default.
• W3212259012 hasConcept C165801399 @default.
• W3212259012 hasConcept C190470478 @default.
• W3212259012 hasConcept C2524010 @default.
• W3212259012 hasConcept C33923547 @default.
• W3212259012 hasConcept C37914503 @default.
• W3212259012 hasConcept C41008148 @default.
• W3212259012 hasConcept C62520636 @default.
• W3212259012 hasConcept C66322947 @default.
• W3212259012 hasConcept C80444323 @default.
• W3212259012 hasConceptScore W3212259012C11413529 @default.
• W3212259012 hasConceptScore W3212259012C114614502 @default.
• W3212259012 hasConceptScore W3212259012C118615104 @default.
• W3212259012 hasConceptScore W3212259012C121332964 @default.
• W3212259012 hasConceptScore W3212259012C129844170 @default.
• W3212259012 hasConceptScore W3212259012C165801399 @default.
• W3212259012 hasConceptScore W3212259012C190470478 @default.
• W3212259012 hasConceptScore W3212259012C2524010 @default.
• W3212259012 hasConceptScore W3212259012C33923547 @default.
• W3212259012 hasConceptScore W3212259012C37914503 @default.
• W3212259012 hasConceptScore W3212259012C41008148 @default.
• W3212259012 hasConceptScore W3212259012C62520636 @default.
• W3212259012 hasConceptScore W3212259012C66322947 @default.
• W3212259012 hasConceptScore W3212259012C80444323 @default.
• W3212259012 hasLocation W32122590121 @default.
• W3212259012 hasOpenAccess W3212259012 @default.
• W3212259012 hasPrimaryLocation W32122590121 @default.
• W3212259012 hasRelatedWork W1522171595 @default.
• W3212259012 hasRelatedWork W1570588782 @default.
• W3212259012 hasRelatedWork W1770127481 @default.
• W3212259012 hasRelatedWork W2008013300 @default.
• W3212259012 hasRelatedWork W2084471831 @default.
• W3212259012 hasRelatedWork W2090339428 @default.
• W3212259012 hasRelatedWork W2497890598 @default.
• W3212259012 hasRelatedWork W2766275246 @default.
• W3212259012 hasRelatedWork W2774158340 @default.
• W3212259012 hasRelatedWork W2801283211 @default.
• W3212259012 hasRelatedWork W2900482350 @default.
• W3212259012 hasRelatedWork W2950428485 @default.
• W3212259012 hasRelatedWork W3003329734 @default.
• W3212259012 hasRelatedWork W3036315659 @default.
• W3212259012 hasRelatedWork W3080569475 @default.
• W3212259012 hasRelatedWork W3082952831 @default.

• next