Matches in SemOpenAlex for { <https://semopenalex.org/work/W4288284003> ?p ?o ?g. }
Showing items 1 to 45 of
45
with 100 items per page.
- W4288284003 abstract "This paper introduces a structured memory which can be easily integrated into a neural network. The memory is very large by design and significantly increases the capacity of the architecture, by up to a billion parameters with a negligible computational overhead. Its design and access pattern is based on product keys, which enable fast and exact nearest neighbor search. The ability to increase the number of parameters while keeping the same computational budget lets the overall system strike a better trade-off between prediction accuracy and computation efficiency both at training and test time. This memory layer allows us to tackle very large scale language modeling tasks. In our experiments we consider a dataset with up to 30 billion words, and we plug our memory layer in a state-of-the-art transformer-based architecture. In particular, we found that a memory augmented model with only 12 layers outperforms a baseline transformer model with 24 layers, while being twice faster at inference time. We release our code for reproducibility purposes." @default.
- W4288284003 created "2022-07-28" @default.
- W4288284003 creator A5027101473 @default.
- W4288284003 creator A5033867133 @default.
- W4288284003 creator A5054371148 @default.
- W4288284003 creator A5067991583 @default.
- W4288284003 creator A5077887756 @default.
- W4288284003 date "2019-12-08" @default.
- W4288284003 modified "2023-10-03" @default.
- W4288284003 title "Large Memory Layers with Product Keys" @default.
- W4288284003 hasPublicationYear "2019" @default.
- W4288284003 type Work @default.
- W4288284003 citedByCount "0" @default.
- W4288284003 crossrefType "proceedings-article" @default.
- W4288284003 hasAuthorship W4288284003A5027101473 @default.
- W4288284003 hasAuthorship W4288284003A5033867133 @default.
- W4288284003 hasAuthorship W4288284003A5054371148 @default.
- W4288284003 hasAuthorship W4288284003A5067991583 @default.
- W4288284003 hasAuthorship W4288284003A5077887756 @default.
- W4288284003 hasBestOaLocation W42882840032 @default.
- W4288284003 hasConcept C2524010 @default.
- W4288284003 hasConcept C33923547 @default.
- W4288284003 hasConcept C41008148 @default.
- W4288284003 hasConcept C90673727 @default.
- W4288284003 hasConceptScore W4288284003C2524010 @default.
- W4288284003 hasConceptScore W4288284003C33923547 @default.
- W4288284003 hasConceptScore W4288284003C41008148 @default.
- W4288284003 hasConceptScore W4288284003C90673727 @default.
- W4288284003 hasLocation W42882840031 @default.
- W4288284003 hasLocation W42882840032 @default.
- W4288284003 hasOpenAccess W4288284003 @default.
- W4288284003 hasPrimaryLocation W42882840031 @default.
- W4288284003 hasRelatedWork W2096946506 @default.
- W4288284003 hasRelatedWork W2130043461 @default.
- W4288284003 hasRelatedWork W2350741829 @default.
- W4288284003 hasRelatedWork W2358668433 @default.
- W4288284003 hasRelatedWork W2376932109 @default.
- W4288284003 hasRelatedWork W2382290278 @default.
- W4288284003 hasRelatedWork W2390279801 @default.
- W4288284003 hasRelatedWork W2748952813 @default.
- W4288284003 hasRelatedWork W2899084033 @default.
- W4288284003 hasRelatedWork W3004735627 @default.
- W4288284003 isParatext "false" @default.
- W4288284003 isRetracted "false" @default.
- W4288284003 workType "article" @default.