Matches in SemOpenAlex for { <https://semopenalex.org/work/W4306384711> ?p ?o ?g. }
Showing items 1 to 67 of
67
with 100 items per page.
- W4306384711 abstract "Transformer-based models are widely used in natural language understanding (NLU) tasks, and multimodal transformers have been effective in visual-language tasks. This study explores distilling visual information from pretrained multimodal transformers to pretrained language encoders. Our framework is inspired by cross-modal encoders' success in visual-language tasks while we alter the learning objective to cater to the language-heavy characteristics of NLU. After training with a small number of extra adapting steps and finetuned, the proposed XDBERT (cross-modal distilled BERT) outperforms pretrained-BERT in general language understanding evaluation (GLUE), situations with adversarial generations (SWAG) benchmarks, and readability benchmarks. We analyze the performance of XDBERT on GLUE to show that the improvement is likely visually grounded." @default.
- W4306384711 created "2022-10-17" @default.
- W4306384711 creator A5040508737 @default.
- W4306384711 creator A5057697869 @default.
- W4306384711 creator A5059902893 @default.
- W4306384711 date "2022-04-14" @default.
- W4306384711 modified "2023-09-28" @default.
- W4306384711 title "XDBERT: Distilling Visual Information to BERT from Cross-Modal Systems to Improve Language Understanding" @default.
- W4306384711 doi "https://doi.org/10.48550/arxiv.2204.07316" @default.
- W4306384711 hasPublicationYear "2022" @default.
- W4306384711 type Work @default.
- W4306384711 citedByCount "0" @default.
- W4306384711 crossrefType "posted-content" @default.
- W4306384711 hasAuthorship W4306384711A5040508737 @default.
- W4306384711 hasAuthorship W4306384711A5057697869 @default.
- W4306384711 hasAuthorship W4306384711A5059902893 @default.
- W4306384711 hasBestOaLocation W43063847111 @default.
- W4306384711 hasConcept C107457646 @default.
- W4306384711 hasConcept C111919701 @default.
- W4306384711 hasConcept C118505674 @default.
- W4306384711 hasConcept C121332964 @default.
- W4306384711 hasConcept C137293760 @default.
- W4306384711 hasConcept C154945302 @default.
- W4306384711 hasConcept C165801399 @default.
- W4306384711 hasConcept C185592680 @default.
- W4306384711 hasConcept C188027245 @default.
- W4306384711 hasConcept C195324797 @default.
- W4306384711 hasConcept C204321447 @default.
- W4306384711 hasConcept C2779439875 @default.
- W4306384711 hasConcept C2983448237 @default.
- W4306384711 hasConcept C41008148 @default.
- W4306384711 hasConcept C62520636 @default.
- W4306384711 hasConcept C66322947 @default.
- W4306384711 hasConcept C71139939 @default.
- W4306384711 hasConceptScore W4306384711C107457646 @default.
- W4306384711 hasConceptScore W4306384711C111919701 @default.
- W4306384711 hasConceptScore W4306384711C118505674 @default.
- W4306384711 hasConceptScore W4306384711C121332964 @default.
- W4306384711 hasConceptScore W4306384711C137293760 @default.
- W4306384711 hasConceptScore W4306384711C154945302 @default.
- W4306384711 hasConceptScore W4306384711C165801399 @default.
- W4306384711 hasConceptScore W4306384711C185592680 @default.
- W4306384711 hasConceptScore W4306384711C188027245 @default.
- W4306384711 hasConceptScore W4306384711C195324797 @default.
- W4306384711 hasConceptScore W4306384711C204321447 @default.
- W4306384711 hasConceptScore W4306384711C2779439875 @default.
- W4306384711 hasConceptScore W4306384711C2983448237 @default.
- W4306384711 hasConceptScore W4306384711C41008148 @default.
- W4306384711 hasConceptScore W4306384711C62520636 @default.
- W4306384711 hasConceptScore W4306384711C66322947 @default.
- W4306384711 hasConceptScore W4306384711C71139939 @default.
- W4306384711 hasLocation W43063847111 @default.
- W4306384711 hasOpenAccess W4306384711 @default.
- W4306384711 hasPrimaryLocation W43063847111 @default.
- W4306384711 hasRelatedWork W2960591372 @default.
- W4306384711 hasRelatedWork W2966892770 @default.
- W4306384711 hasRelatedWork W2995923603 @default.
- W4306384711 hasRelatedWork W3175797870 @default.
- W4306384711 hasRelatedWork W3177381119 @default.
- W4306384711 hasRelatedWork W3207932232 @default.
- W4306384711 hasRelatedWork W4237966389 @default.
- W4306384711 hasRelatedWork W4239973675 @default.
- W4306384711 hasRelatedWork W4283029779 @default.
- W4306384711 hasRelatedWork W4312260499 @default.
- W4306384711 isParatext "false" @default.
- W4306384711 isRetracted "false" @default.
- W4306384711 workType "article" @default.