Matches in SemOpenAlex for { <https://semopenalex.org/work/W4385570209> ?p ?o ?g. }
Showing items 1 to 69 of
69
with 100 items per page.
- W4385570209 abstract "Fusion-in-Decoder (FiD) is a powerful retrieval-augmented language model that sets the state-of-the-art on many knowledge-intensive NLP tasks. However, the architecture used for FiD was chosen by making minimal modifications to a standard T5 model, which our analysis shows to be highly suboptimal for a retrieval-augmented model. In particular, FiD allocates the bulk of FLOPs to the encoder, while the majority of inference time results from memory bandwidth constraints in the decoder. We propose two simple changes to the FiD architecture to alleviate memory bandwidth constraints, and speed up inference by 7x. This allows us to use a much larger decoder at modest cost. We denote FiD with the above modifications as FiDO, and show that it strongly improves performance over existing FiD models for a wide range of inference budgets. For example, FiDO-Large-XXL performs faster inference than FiD-Base and achieves better performance than FiD-Large." @default.
- W4385570209 created "2023-08-05" @default.
- W4385570209 creator A5012884492 @default.
- W4385570209 creator A5021516103 @default.
- W4385570209 creator A5021943393 @default.
- W4385570209 creator A5036319063 @default.
- W4385570209 creator A5051617344 @default.
- W4385570209 creator A5072605113 @default.
- W4385570209 creator A5073439637 @default.
- W4385570209 date "2023-01-01" @default.
- W4385570209 modified "2023-10-18" @default.
- W4385570209 title "FiDO: Fusion-in-Decoder optimized for stronger performance and faster inference" @default.
- W4385570209 doi "https://doi.org/10.18653/v1/2023.findings-acl.732" @default.
- W4385570209 hasPublicationYear "2023" @default.
- W4385570209 type Work @default.
- W4385570209 citedByCount "1" @default.
- W4385570209 countsByYear W43855702092023 @default.
- W4385570209 crossrefType "proceedings-article" @default.
- W4385570209 hasAuthorship W4385570209A5012884492 @default.
- W4385570209 hasAuthorship W4385570209A5021516103 @default.
- W4385570209 hasAuthorship W4385570209A5021943393 @default.
- W4385570209 hasAuthorship W4385570209A5036319063 @default.
- W4385570209 hasAuthorship W4385570209A5051617344 @default.
- W4385570209 hasAuthorship W4385570209A5072605113 @default.
- W4385570209 hasAuthorship W4385570209A5073439637 @default.
- W4385570209 hasBestOaLocation W43855702091 @default.
- W4385570209 hasConcept C111919701 @default.
- W4385570209 hasConcept C118505674 @default.
- W4385570209 hasConcept C123657996 @default.
- W4385570209 hasConcept C142362112 @default.
- W4385570209 hasConcept C153349607 @default.
- W4385570209 hasConcept C154945302 @default.
- W4385570209 hasConcept C173608175 @default.
- W4385570209 hasConcept C188045654 @default.
- W4385570209 hasConcept C2776214188 @default.
- W4385570209 hasConcept C2776257435 @default.
- W4385570209 hasConcept C31258907 @default.
- W4385570209 hasConcept C41008148 @default.
- W4385570209 hasConcept C46743427 @default.
- W4385570209 hasConceptScore W4385570209C111919701 @default.
- W4385570209 hasConceptScore W4385570209C118505674 @default.
- W4385570209 hasConceptScore W4385570209C123657996 @default.
- W4385570209 hasConceptScore W4385570209C142362112 @default.
- W4385570209 hasConceptScore W4385570209C153349607 @default.
- W4385570209 hasConceptScore W4385570209C154945302 @default.
- W4385570209 hasConceptScore W4385570209C173608175 @default.
- W4385570209 hasConceptScore W4385570209C188045654 @default.
- W4385570209 hasConceptScore W4385570209C2776214188 @default.
- W4385570209 hasConceptScore W4385570209C2776257435 @default.
- W4385570209 hasConceptScore W4385570209C31258907 @default.
- W4385570209 hasConceptScore W4385570209C41008148 @default.
- W4385570209 hasConceptScore W4385570209C46743427 @default.
- W4385570209 hasLocation W43855702091 @default.
- W4385570209 hasLocation W43855702092 @default.
- W4385570209 hasOpenAccess W4385570209 @default.
- W4385570209 hasPrimaryLocation W43855702091 @default.
- W4385570209 hasRelatedWork W2057057690 @default.
- W4385570209 hasRelatedWork W2275988210 @default.
- W4385570209 hasRelatedWork W2358964818 @default.
- W4385570209 hasRelatedWork W2359535128 @default.
- W4385570209 hasRelatedWork W2366977406 @default.
- W4385570209 hasRelatedWork W2368184788 @default.
- W4385570209 hasRelatedWork W2381332051 @default.
- W4385570209 hasRelatedWork W2385621972 @default.
- W4385570209 hasRelatedWork W4311992636 @default.
- W4385570209 hasRelatedWork W4385570209 @default.
- W4385570209 isParatext "false" @default.
- W4385570209 isRetracted "false" @default.
- W4385570209 workType "article" @default.