Matches in SemOpenAlex for { <https://semopenalex.org/work/W4385820266> ?p ?o ?g. }
Showing items 1 to 75 of
75
with 100 items per page.
- W4385820266 endingPage "64" @default.
- W4385820266 startingPage "47" @default.
- W4385820266 abstract "Since their release, Transformers have revolutionized many fields from Natural Language Understanding to Computer Vision. Document Understanding (DU) was not left behind with first Transformer based models for DU dating from late 2019. However, the computational complexity of the self-attention operation limits their capabilities to small sequences. In this paper we explore multiple strategies to apply Transformer based models to long multi-page documents. We introduce 2 new multi-modal (text + layout) long-range models for DU. They are based on efficient implementations of Transformers for long sequences. Long-range models can process whole documents at once effectively and are less impaired by the document's length. We compare them to LayoutLM, a classical Transformer adapted for DU and pre-trained on millions of documents. We further propose 2D relative attention bias to guide self-attention towards relevant tokens without harming model efficiency. We observe improvements on multi-page business documents on Information Retrieval for a small performance cost on smaller sequences. Relative 2D attention revealed to be effective on dense text for both normal and long-range models." @default.
- W4385820266 created "2023-08-15" @default.
- W4385820266 creator A5038916044 @default.
- W4385820266 creator A5044289736 @default.
- W4385820266 creator A5057371879 @default.
- W4385820266 creator A5065053376 @default.
- W4385820266 date "2023-01-01" @default.
- W4385820266 modified "2023-10-03" @default.
- W4385820266 title "Long-Range Transformer Architectures for Document Understanding" @default.
- W4385820266 cites W1966382373 @default.
- W4385820266 cites W2064675550 @default.
- W4385820266 cites W2250539671 @default.
- W4385820266 cites W2296283641 @default.
- W4385820266 cites W2891117443 @default.
- W4385820266 cites W2923014074 @default.
- W4385820266 cites W2962739339 @default.
- W4385820266 cites W2962772269 @default.
- W4385820266 cites W2963341956 @default.
- W4385820266 cites W2963925437 @default.
- W4385820266 cites W3003484198 @default.
- W4385820266 cites W3104953317 @default.
- W4385820266 cites W3105238007 @default.
- W4385820266 cites W3113753692 @default.
- W4385820266 cites W3120043490 @default.
- W4385820266 cites W3176851559 @default.
- W4385820266 cites W3202839357 @default.
- W4385820266 cites W4225525539 @default.
- W4385820266 cites W4304013646 @default.
- W4385820266 doi "https://doi.org/10.1007/978-3-031-41501-2_4" @default.
- W4385820266 hasPublicationYear "2023" @default.
- W4385820266 type Work @default.
- W4385820266 citedByCount "0" @default.
- W4385820266 crossrefType "book-chapter" @default.
- W4385820266 hasAuthorship W4385820266A5038916044 @default.
- W4385820266 hasAuthorship W4385820266A5044289736 @default.
- W4385820266 hasAuthorship W4385820266A5057371879 @default.
- W4385820266 hasAuthorship W4385820266A5065053376 @default.
- W4385820266 hasBestOaLocation W43858202662 @default.
- W4385820266 hasConcept C115903868 @default.
- W4385820266 hasConcept C119599485 @default.
- W4385820266 hasConcept C127413603 @default.
- W4385820266 hasConcept C154945302 @default.
- W4385820266 hasConcept C165801399 @default.
- W4385820266 hasConcept C204321447 @default.
- W4385820266 hasConcept C26713055 @default.
- W4385820266 hasConcept C41008148 @default.
- W4385820266 hasConcept C66322947 @default.
- W4385820266 hasConceptScore W4385820266C115903868 @default.
- W4385820266 hasConceptScore W4385820266C119599485 @default.
- W4385820266 hasConceptScore W4385820266C127413603 @default.
- W4385820266 hasConceptScore W4385820266C154945302 @default.
- W4385820266 hasConceptScore W4385820266C165801399 @default.
- W4385820266 hasConceptScore W4385820266C204321447 @default.
- W4385820266 hasConceptScore W4385820266C26713055 @default.
- W4385820266 hasConceptScore W4385820266C41008148 @default.
- W4385820266 hasConceptScore W4385820266C66322947 @default.
- W4385820266 hasLocation W43858202661 @default.
- W4385820266 hasLocation W43858202662 @default.
- W4385820266 hasOpenAccess W4385820266 @default.
- W4385820266 hasPrimaryLocation W43858202661 @default.
- W4385820266 hasRelatedWork W1485630101 @default.
- W4385820266 hasRelatedWork W1596157658 @default.
- W4385820266 hasRelatedWork W1824798286 @default.
- W4385820266 hasRelatedWork W1964111820 @default.
- W4385820266 hasRelatedWork W2063727421 @default.
- W4385820266 hasRelatedWork W2083338789 @default.
- W4385820266 hasRelatedWork W2112690207 @default.
- W4385820266 hasRelatedWork W2741796204 @default.
- W4385820266 hasRelatedWork W4244853187 @default.
- W4385820266 hasRelatedWork W1501086624 @default.
- W4385820266 isParatext "false" @default.
- W4385820266 isRetracted "false" @default.
- W4385820266 workType "book-chapter" @default.