Matches in SemOpenAlex for { <https://semopenalex.org/work/W4387222181> ?p ?o ?g. }
Showing items 1 to 59 of
59
with 100 items per page.
- W4387222181 abstract "We present ReCAT, a recursive composition augmented Transformer that is able to explicitly model hierarchical syntactic structures of raw texts without relying on gold trees during both learning and inference. Existing research along this line restricts data to follow a hierarchical tree structure and thus lacks inter-span communications. To overcome the problem, we propose a novel contextual inside-outside (CIO) layer that learns contextualized representations of spans through bottom-up and top-down passes, where a bottom-up pass forms representations of high-level spans by composing low-level spans, while a top-down pass combines information inside and outside a span. By stacking several CIO layers between the embedding layer and the attention layers in Transformer, the ReCAT model can perform both deep intra-span and deep inter-span interactions, and thus generate multi-grained representations fully contextualized with other spans. Moreover, the CIO layers can be jointly pre-trained with Transformers, making ReCAT enjoy scaling ability, strong performance, and interpretability at the same time. We conduct experiments on various sentence-level and span-level tasks. Evaluation results indicate that ReCAT can significantly outperform vanilla Transformer models on all span-level tasks and baselines that combine recursive networks with Transformers on natural language inference tasks. More interestingly, the hierarchical structures induced by ReCAT exhibit strong consistency with human-annotated syntactic trees, indicating good interpretability brought by the CIO layers." @default.
- W4387222181 created "2023-10-01" @default.
- W4387222181 creator A5004605359 @default.
- W4387222181 creator A5031485135 @default.
- W4387222181 creator A5046744673 @default.
- W4387222181 creator A5072165642 @default.
- W4387222181 date "2023-09-28" @default.
- W4387222181 modified "2023-10-01" @default.
- W4387222181 title "Augmenting transformers with recursively composed multi-grained representations" @default.
- W4387222181 doi "https://doi.org/10.48550/arxiv.2309.16319" @default.
- W4387222181 hasPublicationYear "2023" @default.
- W4387222181 type Work @default.
- W4387222181 citedByCount "0" @default.
- W4387222181 crossrefType "posted-content" @default.
- W4387222181 hasAuthorship W4387222181A5004605359 @default.
- W4387222181 hasAuthorship W4387222181A5031485135 @default.
- W4387222181 hasAuthorship W4387222181A5046744673 @default.
- W4387222181 hasAuthorship W4387222181A5072165642 @default.
- W4387222181 hasBestOaLocation W43872221811 @default.
- W4387222181 hasConcept C121332964 @default.
- W4387222181 hasConcept C154945302 @default.
- W4387222181 hasConcept C165801399 @default.
- W4387222181 hasConcept C204321447 @default.
- W4387222181 hasConcept C2776214188 @default.
- W4387222181 hasConcept C2777530160 @default.
- W4387222181 hasConcept C2781067378 @default.
- W4387222181 hasConcept C33347731 @default.
- W4387222181 hasConcept C41008148 @default.
- W4387222181 hasConcept C46141821 @default.
- W4387222181 hasConcept C62520636 @default.
- W4387222181 hasConcept C66322947 @default.
- W4387222181 hasConceptScore W4387222181C121332964 @default.
- W4387222181 hasConceptScore W4387222181C154945302 @default.
- W4387222181 hasConceptScore W4387222181C165801399 @default.
- W4387222181 hasConceptScore W4387222181C204321447 @default.
- W4387222181 hasConceptScore W4387222181C2776214188 @default.
- W4387222181 hasConceptScore W4387222181C2777530160 @default.
- W4387222181 hasConceptScore W4387222181C2781067378 @default.
- W4387222181 hasConceptScore W4387222181C33347731 @default.
- W4387222181 hasConceptScore W4387222181C41008148 @default.
- W4387222181 hasConceptScore W4387222181C46141821 @default.
- W4387222181 hasConceptScore W4387222181C62520636 @default.
- W4387222181 hasConceptScore W4387222181C66322947 @default.
- W4387222181 hasLocation W43872221811 @default.
- W4387222181 hasOpenAccess W4387222181 @default.
- W4387222181 hasPrimaryLocation W43872221811 @default.
- W4387222181 hasRelatedWork W159132833 @default.
- W4387222181 hasRelatedWork W2947762898 @default.
- W4387222181 hasRelatedWork W2968260065 @default.
- W4387222181 hasRelatedWork W4206534706 @default.
- W4387222181 hasRelatedWork W4229035991 @default.
- W4387222181 hasRelatedWork W4307308917 @default.
- W4387222181 hasRelatedWork W4318978824 @default.
- W4387222181 hasRelatedWork W4385573168 @default.
- W4387222181 hasRelatedWork W4385873483 @default.
- W4387222181 hasRelatedWork W4385877744 @default.
- W4387222181 isParatext "false" @default.
- W4387222181 isRetracted "false" @default.
- W4387222181 workType "article" @default.