Matches in SemOpenAlex for { <https://semopenalex.org/work/W4320166805> ?p ?o ?g. }
Showing items 1 to 95 of
95
with 100 items per page.
- W4320166805 abstract "Retrieval-augmented language models such as Fusion-in-Decoder are powerful, setting the state of the art on a variety of knowledge-intensive tasks. However, they are also expensive, due to the need to encode a large number of retrieved passages. Some work avoids this cost by pre-encoding a text corpus into a memory and retrieving dense representations directly. However, pre-encoding memory incurs a severe quality penalty as the memory representations are not conditioned on the current input. We propose LUMEN, a hybrid between these two extremes, pre-computing the majority of the retrieval representation and completing the encoding on the fly using a live encoder that is conditioned on the question and fine-tuned for the task. We show that LUMEN significantly outperforms pure memory on multiple question-answering tasks while being much cheaper than FiD, and outperforms both for any given compute budget. Moreover, the advantage of LUMEN over FiD increases with model size." @default.
- W4320166805 created "2023-02-13" @default.
- W4320166805 creator A5012884492 @default.
- W4320166805 creator A5021516103 @default.
- W4320166805 creator A5021943393 @default.
- W4320166805 creator A5036319063 @default.
- W4320166805 creator A5051617344 @default.
- W4320166805 creator A5072605113 @default.
- W4320166805 creator A5073439637 @default.
- W4320166805 date "2023-01-25" @default.
- W4320166805 modified "2023-10-18" @default.
- W4320166805 title "Pre-computed memory or on-the-fly encoding? A hybrid approach to retrieval augmentation makes the most of your compute" @default.
- W4320166805 doi "https://doi.org/10.48550/arxiv.2301.10448" @default.
- W4320166805 hasPublicationYear "2023" @default.
- W4320166805 type Work @default.
- W4320166805 citedByCount "0" @default.
- W4320166805 crossrefType "posted-content" @default.
- W4320166805 hasAuthorship W4320166805A5012884492 @default.
- W4320166805 hasAuthorship W4320166805A5021516103 @default.
- W4320166805 hasAuthorship W4320166805A5021943393 @default.
- W4320166805 hasAuthorship W4320166805A5036319063 @default.
- W4320166805 hasAuthorship W4320166805A5051617344 @default.
- W4320166805 hasAuthorship W4320166805A5072605113 @default.
- W4320166805 hasAuthorship W4320166805A5073439637 @default.
- W4320166805 hasBestOaLocation W43201668051 @default.
- W4320166805 hasConcept C104317684 @default.
- W4320166805 hasConcept C111472728 @default.
- W4320166805 hasConcept C111919701 @default.
- W4320166805 hasConcept C118505674 @default.
- W4320166805 hasConcept C125411270 @default.
- W4320166805 hasConcept C134306372 @default.
- W4320166805 hasConcept C138885662 @default.
- W4320166805 hasConcept C154945302 @default.
- W4320166805 hasConcept C162324750 @default.
- W4320166805 hasConcept C177264268 @default.
- W4320166805 hasConcept C17744445 @default.
- W4320166805 hasConcept C185592680 @default.
- W4320166805 hasConcept C187736073 @default.
- W4320166805 hasConcept C199360897 @default.
- W4320166805 hasConcept C199539241 @default.
- W4320166805 hasConcept C2776359362 @default.
- W4320166805 hasConcept C2776760102 @default.
- W4320166805 hasConcept C2779530757 @default.
- W4320166805 hasConcept C2780451532 @default.
- W4320166805 hasConcept C2781020372 @default.
- W4320166805 hasConcept C33923547 @default.
- W4320166805 hasConcept C41008148 @default.
- W4320166805 hasConcept C44291984 @default.
- W4320166805 hasConcept C55493867 @default.
- W4320166805 hasConcept C66746571 @default.
- W4320166805 hasConcept C77618280 @default.
- W4320166805 hasConcept C94625758 @default.
- W4320166805 hasConceptScore W4320166805C104317684 @default.
- W4320166805 hasConceptScore W4320166805C111472728 @default.
- W4320166805 hasConceptScore W4320166805C111919701 @default.
- W4320166805 hasConceptScore W4320166805C118505674 @default.
- W4320166805 hasConceptScore W4320166805C125411270 @default.
- W4320166805 hasConceptScore W4320166805C134306372 @default.
- W4320166805 hasConceptScore W4320166805C138885662 @default.
- W4320166805 hasConceptScore W4320166805C154945302 @default.
- W4320166805 hasConceptScore W4320166805C162324750 @default.
- W4320166805 hasConceptScore W4320166805C177264268 @default.
- W4320166805 hasConceptScore W4320166805C17744445 @default.
- W4320166805 hasConceptScore W4320166805C185592680 @default.
- W4320166805 hasConceptScore W4320166805C187736073 @default.
- W4320166805 hasConceptScore W4320166805C199360897 @default.
- W4320166805 hasConceptScore W4320166805C199539241 @default.
- W4320166805 hasConceptScore W4320166805C2776359362 @default.
- W4320166805 hasConceptScore W4320166805C2776760102 @default.
- W4320166805 hasConceptScore W4320166805C2779530757 @default.
- W4320166805 hasConceptScore W4320166805C2780451532 @default.
- W4320166805 hasConceptScore W4320166805C2781020372 @default.
- W4320166805 hasConceptScore W4320166805C33923547 @default.
- W4320166805 hasConceptScore W4320166805C41008148 @default.
- W4320166805 hasConceptScore W4320166805C44291984 @default.
- W4320166805 hasConceptScore W4320166805C55493867 @default.
- W4320166805 hasConceptScore W4320166805C66746571 @default.
- W4320166805 hasConceptScore W4320166805C77618280 @default.
- W4320166805 hasConceptScore W4320166805C94625758 @default.
- W4320166805 hasLocation W43201668051 @default.
- W4320166805 hasOpenAccess W4320166805 @default.
- W4320166805 hasPrimaryLocation W43201668051 @default.
- W4320166805 hasRelatedWork W140709781 @default.
- W4320166805 hasRelatedWork W1995170466 @default.
- W4320166805 hasRelatedWork W2076903247 @default.
- W4320166805 hasRelatedWork W2359640100 @default.
- W4320166805 hasRelatedWork W2375209648 @default.
- W4320166805 hasRelatedWork W2389073067 @default.
- W4320166805 hasRelatedWork W3212159072 @default.
- W4320166805 hasRelatedWork W4226181950 @default.
- W4320166805 hasRelatedWork W4320166805 @default.
- W4320166805 hasRelatedWork W4378767251 @default.
- W4320166805 isParatext "false" @default.
- W4320166805 isRetracted "false" @default.
- W4320166805 workType "article" @default.