Matches in SemOpenAlex for { <https://semopenalex.org/work/W4378505213> ?p ?o ?g. }
Showing items 1 to 64 of
64
with 100 items per page.
- W4378505213 abstract "Augmenting language models with a retrieval mechanism has been shown to significantly improve their performance while keeping the number of parameters low. Retrieval-augmented models commonly rely on a semantic retrieval mechanism based on the similarity between dense representations of the query chunk and potential neighbors. In this paper, we study the state-of-the-art Retro model and observe that its performance gain is better explained by surface-level similarities, such as token overlap. Inspired by this, we replace the semantic retrieval in Retro with a surface-level method based on BM25, obtaining a significant reduction in perplexity. As full BM25 retrieval can be computationally costly for large datasets, we also apply it in a re-ranking scenario, gaining part of the perplexity reduction with minimal computational overhead." @default.
- W4378505213 created "2023-05-27" @default.
- W4378505213 creator A5054122324 @default.
- W4378505213 creator A5069485198 @default.
- W4378505213 creator A5073142079 @default.
- W4378505213 creator A5079881269 @default.
- W4378505213 date "2023-05-25" @default.
- W4378505213 modified "2023-10-18" @default.
- W4378505213 title "Surface-Based Retrieval Reduces Perplexity of Retrieval-Augmented Language Models" @default.
- W4378505213 doi "https://doi.org/10.48550/arxiv.2305.16243" @default.
- W4378505213 hasPublicationYear "2023" @default.
- W4378505213 type Work @default.
- W4378505213 citedByCount "0" @default.
- W4378505213 crossrefType "posted-content" @default.
- W4378505213 hasAuthorship W4378505213A5054122324 @default.
- W4378505213 hasAuthorship W4378505213A5069485198 @default.
- W4378505213 hasAuthorship W4378505213A5073142079 @default.
- W4378505213 hasAuthorship W4378505213A5079881269 @default.
- W4378505213 hasBestOaLocation W43785052131 @default.
- W4378505213 hasConcept C100279451 @default.
- W4378505213 hasConcept C103278499 @default.
- W4378505213 hasConcept C111335779 @default.
- W4378505213 hasConcept C115961682 @default.
- W4378505213 hasConcept C137293760 @default.
- W4378505213 hasConcept C154945302 @default.
- W4378505213 hasConcept C189430467 @default.
- W4378505213 hasConcept C204321447 @default.
- W4378505213 hasConcept C23123220 @default.
- W4378505213 hasConcept C2524010 @default.
- W4378505213 hasConcept C33923547 @default.
- W4378505213 hasConcept C38652104 @default.
- W4378505213 hasConcept C41008148 @default.
- W4378505213 hasConcept C48145219 @default.
- W4378505213 hasConceptScore W4378505213C100279451 @default.
- W4378505213 hasConceptScore W4378505213C103278499 @default.
- W4378505213 hasConceptScore W4378505213C111335779 @default.
- W4378505213 hasConceptScore W4378505213C115961682 @default.
- W4378505213 hasConceptScore W4378505213C137293760 @default.
- W4378505213 hasConceptScore W4378505213C154945302 @default.
- W4378505213 hasConceptScore W4378505213C189430467 @default.
- W4378505213 hasConceptScore W4378505213C204321447 @default.
- W4378505213 hasConceptScore W4378505213C23123220 @default.
- W4378505213 hasConceptScore W4378505213C2524010 @default.
- W4378505213 hasConceptScore W4378505213C33923547 @default.
- W4378505213 hasConceptScore W4378505213C38652104 @default.
- W4378505213 hasConceptScore W4378505213C41008148 @default.
- W4378505213 hasConceptScore W4378505213C48145219 @default.
- W4378505213 hasLocation W43785052131 @default.
- W4378505213 hasLocation W43785052132 @default.
- W4378505213 hasOpenAccess W4378505213 @default.
- W4378505213 hasPrimaryLocation W43785052131 @default.
- W4378505213 hasRelatedWork W1989705153 @default.
- W4378505213 hasRelatedWork W2131111393 @default.
- W4378505213 hasRelatedWork W2138279922 @default.
- W4378505213 hasRelatedWork W2410936271 @default.
- W4378505213 hasRelatedWork W2496228846 @default.
- W4378505213 hasRelatedWork W2896411932 @default.
- W4378505213 hasRelatedWork W3116220693 @default.
- W4378505213 hasRelatedWork W3153132431 @default.
- W4378505213 hasRelatedWork W4312121705 @default.
- W4378505213 hasRelatedWork W59929963 @default.
- W4378505213 isParatext "false" @default.
- W4378505213 isRetracted "false" @default.
- W4378505213 workType "article" @default.