Matches in SemOpenAlex for { <https://semopenalex.org/work/W3014242124> ?p ?o ?g. }
Showing items 1 to 97 of
97
with 100 items per page.
- W3014242124 abstract "Many applications today, such as NLP, network analysis, and code analysis, rely on semantically embedding objects into low-dimensional fixed-length vectors. Such embeddings naturally provide a way to perform useful downstream tasks, such as identifying relations among objects or predicting objects for a given context, etc. Unfortunately, the training necessary for accurate embeddings is usually computationally intensive and requires processing large amounts of data. Furthermore, distributing this training is challenging. Most embedding training uses stochastic gradient descent (SGD), an inherently sequential algorithm. Prior approaches to parallelizing SGD do not honor these dependencies and thus potentially suffer poor convergence. This paper presents a distributed training framework for a class of applications that use Skip-gram-like models to generate embeddings. We call this class Any2Vec and it includes Word2Vec, DeepWalk, and Node2Vec among others. We first formulate Any2Vec training algorithm as a graph application and leverage the state-of-the-art distributed graph analytics framework, D-Galois. We adapt D-Galois to support dynamic graph generation and repartitioning, and incorporate novel communication optimizations. Finally, we introduce a novel way to combine gradients during distributed training to prevent accuracy loss. We show that our framework, called GraphAny2Vec, matches on a cluster of 32 hosts the accuracy of the state-of-the-art shared-memory implementations of Word2Vec and Vertex2Vec on 1 host, and gives a geo-mean speedup of 12x and 5x respectively. Furthermore, GraphAny2Vec is on average 2x faster than the state-of-the-art distributed Word2Vec implementation, DMTK, on 32 hosts. We also show the superiority of our Gradient Combiner independent of GraphAny2Vec by incorporating it in DMTK, which raises its accuracy by > 30%." @default.
- W3014242124 created "2020-04-10" @default.
- W3014242124 creator A5001454502 @default.
- W3014242124 creator A5011198874 @default.
- W3014242124 creator A5013212213 @default.
- W3014242124 creator A5040636191 @default.
- W3014242124 creator A5056166013 @default.
- W3014242124 creator A5077514991 @default.
- W3014242124 date "2019-09-08" @default.
- W3014242124 modified "2023-09-27" @default.
- W3014242124 title "Distributed Training of Embeddings using Graph Analytics" @default.
- W3014242124 cites W168564468 @default.
- W3014242124 cites W1955857676 @default.
- W3014242124 cites W2132339004 @default.
- W3014242124 cites W2138243089 @default.
- W3014242124 cites W2141599568 @default.
- W3014242124 cites W2157462866 @default.
- W3014242124 cites W2168231600 @default.
- W3014242124 cites W2622263826 @default.
- W3014242124 cites W2798525482 @default.
- W3014242124 cites W2917950505 @default.
- W3014242124 cites W2949547296 @default.
- W3014242124 cites W2950577311 @default.
- W3014242124 cites W2951861246 @default.
- W3014242124 cites W2962715821 @default.
- W3014242124 cites W2962836134 @default.
- W3014242124 cites W2964150020 @default.
- W3014242124 cites W2972120381 @default.
- W3014242124 cites W620279967 @default.
- W3014242124 hasPublicationYear "2019" @default.
- W3014242124 type Work @default.
- W3014242124 sameAs 3014242124 @default.
- W3014242124 citedByCount "0" @default.
- W3014242124 crossrefType "posted-content" @default.
- W3014242124 hasAuthorship W3014242124A5001454502 @default.
- W3014242124 hasAuthorship W3014242124A5011198874 @default.
- W3014242124 hasAuthorship W3014242124A5013212213 @default.
- W3014242124 hasAuthorship W3014242124A5040636191 @default.
- W3014242124 hasAuthorship W3014242124A5056166013 @default.
- W3014242124 hasAuthorship W3014242124A5077514991 @default.
- W3014242124 hasConcept C119857082 @default.
- W3014242124 hasConcept C120314980 @default.
- W3014242124 hasConcept C124101348 @default.
- W3014242124 hasConcept C132525143 @default.
- W3014242124 hasConcept C153083717 @default.
- W3014242124 hasConcept C154945302 @default.
- W3014242124 hasConcept C173608175 @default.
- W3014242124 hasConcept C206688291 @default.
- W3014242124 hasConcept C2776461190 @default.
- W3014242124 hasConcept C41008148 @default.
- W3014242124 hasConcept C41608201 @default.
- W3014242124 hasConcept C50644808 @default.
- W3014242124 hasConcept C68339613 @default.
- W3014242124 hasConcept C79158427 @default.
- W3014242124 hasConcept C80444323 @default.
- W3014242124 hasConceptScore W3014242124C119857082 @default.
- W3014242124 hasConceptScore W3014242124C120314980 @default.
- W3014242124 hasConceptScore W3014242124C124101348 @default.
- W3014242124 hasConceptScore W3014242124C132525143 @default.
- W3014242124 hasConceptScore W3014242124C153083717 @default.
- W3014242124 hasConceptScore W3014242124C154945302 @default.
- W3014242124 hasConceptScore W3014242124C173608175 @default.
- W3014242124 hasConceptScore W3014242124C206688291 @default.
- W3014242124 hasConceptScore W3014242124C2776461190 @default.
- W3014242124 hasConceptScore W3014242124C41008148 @default.
- W3014242124 hasConceptScore W3014242124C41608201 @default.
- W3014242124 hasConceptScore W3014242124C50644808 @default.
- W3014242124 hasConceptScore W3014242124C68339613 @default.
- W3014242124 hasConceptScore W3014242124C79158427 @default.
- W3014242124 hasConceptScore W3014242124C80444323 @default.
- W3014242124 hasLocation W30142421241 @default.
- W3014242124 hasOpenAccess W3014242124 @default.
- W3014242124 hasPrimaryLocation W30142421241 @default.
- W3014242124 hasRelatedWork W2404374318 @default.
- W3014242124 hasRelatedWork W2734338834 @default.
- W3014242124 hasRelatedWork W2756413773 @default.
- W3014242124 hasRelatedWork W2890855364 @default.
- W3014242124 hasRelatedWork W2896448469 @default.
- W3014242124 hasRelatedWork W2914514330 @default.
- W3014242124 hasRelatedWork W2953484813 @default.
- W3014242124 hasRelatedWork W2961882312 @default.
- W3014242124 hasRelatedWork W2963660704 @default.
- W3014242124 hasRelatedWork W2965115497 @default.
- W3014242124 hasRelatedWork W2971894730 @default.
- W3014242124 hasRelatedWork W2998900284 @default.
- W3014242124 hasRelatedWork W3021651910 @default.
- W3014242124 hasRelatedWork W3083866331 @default.
- W3014242124 hasRelatedWork W3104211877 @default.
- W3014242124 hasRelatedWork W3105440105 @default.
- W3014242124 hasRelatedWork W3169777530 @default.
- W3014242124 hasRelatedWork W3174716076 @default.
- W3014242124 hasRelatedWork W3175594345 @default.
- W3014242124 hasRelatedWork W3196464409 @default.
- W3014242124 isParatext "false" @default.
- W3014242124 isRetracted "false" @default.
- W3014242124 magId "3014242124" @default.
- W3014242124 workType "article" @default.