Matches in SemOpenAlex for { <https://semopenalex.org/work/W4380714594> ?p ?o ?g. }
Showing items 1 to 65 of
65
with 100 items per page.
- W4380714594 abstract "Sparse data are common. The traditional ``handcrafted'' features are often sparse. Embedding vectors from trained models can also be very sparse, for example, embeddings trained via the ``ReLu'' activation function. In this paper, we report our exploration of efficient search in sparse data with graph-based ANN algorithms (e.g., HNSW, or SONG which is the GPU version of HNSW), which are popular in industrial practice, e.g., search and ads (advertising). We experiment with the proprietary ads targeting application, as well as benchmark public datasets. For ads targeting, we train embeddings with the standard ``cosine two-tower'' model and we also develop the ``chi-square two-tower'' model. Both models produce (highly) sparse embeddings when they are integrated with the ``ReLu'' activation function. In EBR (embedding-based retrieval) applications, after we the embeddings are trained, the next crucial task is the approximate near neighbor (ANN) search for serving. While there are many ANN algorithms we can choose from, in this study, we focus on the graph-based ANN algorithm (e.g., HNSW-type). Sparse embeddings should help improve the efficiency of EBR. One benefit is the reduced memory cost for the embeddings. The other obvious benefit is the reduced computational time for evaluating similarities, because, for graph-based ANN algorithms such as HNSW, computing similarities is often the dominating cost. In addition to the effort on leveraging data sparsity for storage and computation, we also integrate ``sign cauchy random projections'' (SignCRP) to hash vectors to bits, to further reduce the memory cost and speed up the ANN search. In NIPS'13, SignCRP was proposed to hash the chi-square similarity, which is a well-adopted nonlinear kernel in NLP and computer vision. Therefore, the chi-square two-tower model, SignCRP, and HNSW are now tightly integrated." @default.
- W4380714594 created "2023-06-15" @default.
- W4380714594 creator A5026564599 @default.
- W4380714594 creator A5037787380 @default.
- W4380714594 creator A5055838753 @default.
- W4380714594 creator A5058932360 @default.
- W4380714594 creator A5062226591 @default.
- W4380714594 creator A5073070877 @default.
- W4380714594 date "2023-06-13" @default.
- W4380714594 modified "2023-10-14" @default.
- W4380714594 title "Practice with Graph-based ANN Algorithms on Sparse Data: Chi-square Two-tower model, HNSW, Sign Cauchy Projections" @default.
- W4380714594 doi "https://doi.org/10.48550/arxiv.2306.07607" @default.
- W4380714594 hasPublicationYear "2023" @default.
- W4380714594 type Work @default.
- W4380714594 citedByCount "0" @default.
- W4380714594 crossrefType "posted-content" @default.
- W4380714594 hasAuthorship W4380714594A5026564599 @default.
- W4380714594 hasAuthorship W4380714594A5037787380 @default.
- W4380714594 hasAuthorship W4380714594A5055838753 @default.
- W4380714594 hasAuthorship W4380714594A5058932360 @default.
- W4380714594 hasAuthorship W4380714594A5062226591 @default.
- W4380714594 hasAuthorship W4380714594A5073070877 @default.
- W4380714594 hasBestOaLocation W43807145941 @default.
- W4380714594 hasConcept C102192266 @default.
- W4380714594 hasConcept C11413529 @default.
- W4380714594 hasConcept C13251829 @default.
- W4380714594 hasConcept C132525143 @default.
- W4380714594 hasConcept C13280743 @default.
- W4380714594 hasConcept C154945302 @default.
- W4380714594 hasConcept C185798385 @default.
- W4380714594 hasConcept C203776342 @default.
- W4380714594 hasConcept C205649164 @default.
- W4380714594 hasConcept C41008148 @default.
- W4380714594 hasConcept C41608201 @default.
- W4380714594 hasConcept C45374587 @default.
- W4380714594 hasConcept C80444323 @default.
- W4380714594 hasConceptScore W4380714594C102192266 @default.
- W4380714594 hasConceptScore W4380714594C11413529 @default.
- W4380714594 hasConceptScore W4380714594C13251829 @default.
- W4380714594 hasConceptScore W4380714594C132525143 @default.
- W4380714594 hasConceptScore W4380714594C13280743 @default.
- W4380714594 hasConceptScore W4380714594C154945302 @default.
- W4380714594 hasConceptScore W4380714594C185798385 @default.
- W4380714594 hasConceptScore W4380714594C203776342 @default.
- W4380714594 hasConceptScore W4380714594C205649164 @default.
- W4380714594 hasConceptScore W4380714594C41008148 @default.
- W4380714594 hasConceptScore W4380714594C41608201 @default.
- W4380714594 hasConceptScore W4380714594C45374587 @default.
- W4380714594 hasConceptScore W4380714594C80444323 @default.
- W4380714594 hasLocation W43807145941 @default.
- W4380714594 hasOpenAccess W4380714594 @default.
- W4380714594 hasPrimaryLocation W43807145941 @default.
- W4380714594 hasRelatedWork W1568173680 @default.
- W4380714594 hasRelatedWork W2061746685 @default.
- W4380714594 hasRelatedWork W2923818335 @default.
- W4380714594 hasRelatedWork W3035116611 @default.
- W4380714594 hasRelatedWork W3044158376 @default.
- W4380714594 hasRelatedWork W3044354590 @default.
- W4380714594 hasRelatedWork W4226361842 @default.
- W4380714594 hasRelatedWork W4236163602 @default.
- W4380714594 hasRelatedWork W4287763734 @default.
- W4380714594 hasRelatedWork W4310879833 @default.
- W4380714594 isParatext "false" @default.
- W4380714594 isRetracted "false" @default.
- W4380714594 workType "article" @default.