Matches in SemOpenAlex for { <https://semopenalex.org/work/W4378976162> ?p ?o ?g. }
Showing items 1 to 99 of
99
with 100 items per page.
- W4378976162 endingPage "2232" @default.
- W4378976162 startingPage "2220" @default.
- W4378976162 abstract "We observe that for lip reading, the language is locally transformed, instead of globally transformed, i.e., speaking and writing follow the same basic grammar rules. In this work, we present a cross-modal language model to tackle the lip-reading challenge on silent videos. Compared to previous works, we consider multi-motion-informed contexts composed of multiple lip-motion representations from different subspaces to guide decoding via the source-target attention mechanism. We present a piece-wise pre-training strategy inspired by multi-task learning to pre-train a visual module to generate multi-motioninformed contexts for cross-modality and pre-train a decoder to generate texts for language modeling. Our final large-scale model outperforms baseline models on four datasets: LRS2, LRS3, LRW, and GRID. We will open our source code on GitHub." @default.
- W4378976162 created "2023-06-02" @default.
- W4378976162 creator A5046520540 @default.
- W4378976162 creator A5081756234 @default.
- W4378976162 date "2023-01-01" @default.
- W4378976162 modified "2023-09-29" @default.
- W4378976162 title "Cross-modal Language Modeling in Multi-motion-informed Context for Lip Reading" @default.
- W4378976162 cites W1902237438 @default.
- W4378976162 cites W2015143272 @default.
- W4378976162 cites W2139501017 @default.
- W4378976162 cites W2194775991 @default.
- W4378976162 cites W2243738093 @default.
- W4378976162 cites W2267805933 @default.
- W4378976162 cites W2404704342 @default.
- W4378976162 cites W2594690981 @default.
- W4378976162 cites W2734984521 @default.
- W4378976162 cites W2888779557 @default.
- W4378976162 cites W2890952074 @default.
- W4378976162 cites W2943845043 @default.
- W4378976162 cites W2952746495 @default.
- W4378976162 cites W2962784628 @default.
- W4378976162 cites W2963362078 @default.
- W4378976162 cites W2963407669 @default.
- W4378976162 cites W2963528589 @default.
- W4378976162 cites W2963654155 @default.
- W4378976162 cites W2963785710 @default.
- W4378976162 cites W2963804993 @default.
- W4378976162 cites W2964054038 @default.
- W4378976162 cites W2972756321 @default.
- W4378976162 cites W2981501041 @default.
- W4378976162 cites W2996970093 @default.
- W4378976162 cites W2999528291 @default.
- W4378976162 cites W3015830103 @default.
- W4378976162 cites W3016011581 @default.
- W4378976162 cites W3162293946 @default.
- W4378976162 doi "https://doi.org/10.1109/taslp.2023.3282109" @default.
- W4378976162 hasPublicationYear "2023" @default.
- W4378976162 type Work @default.
- W4378976162 citedByCount "0" @default.
- W4378976162 crossrefType "journal-article" @default.
- W4378976162 hasAuthorship W4378976162A5046520540 @default.
- W4378976162 hasAuthorship W4378976162A5081756234 @default.
- W4378976162 hasConcept C104114177 @default.
- W4378976162 hasConcept C138885662 @default.
- W4378976162 hasConcept C151730666 @default.
- W4378976162 hasConcept C154945302 @default.
- W4378976162 hasConcept C162324750 @default.
- W4378976162 hasConcept C185592680 @default.
- W4378976162 hasConcept C187736073 @default.
- W4378976162 hasConcept C188027245 @default.
- W4378976162 hasConcept C204321447 @default.
- W4378976162 hasConcept C26022165 @default.
- W4378976162 hasConcept C2779343474 @default.
- W4378976162 hasConcept C2780451532 @default.
- W4378976162 hasConcept C41008148 @default.
- W4378976162 hasConcept C41895202 @default.
- W4378976162 hasConcept C554936623 @default.
- W4378976162 hasConcept C70437156 @default.
- W4378976162 hasConcept C71139939 @default.
- W4378976162 hasConcept C86803240 @default.
- W4378976162 hasConceptScore W4378976162C104114177 @default.
- W4378976162 hasConceptScore W4378976162C138885662 @default.
- W4378976162 hasConceptScore W4378976162C151730666 @default.
- W4378976162 hasConceptScore W4378976162C154945302 @default.
- W4378976162 hasConceptScore W4378976162C162324750 @default.
- W4378976162 hasConceptScore W4378976162C185592680 @default.
- W4378976162 hasConceptScore W4378976162C187736073 @default.
- W4378976162 hasConceptScore W4378976162C188027245 @default.
- W4378976162 hasConceptScore W4378976162C204321447 @default.
- W4378976162 hasConceptScore W4378976162C26022165 @default.
- W4378976162 hasConceptScore W4378976162C2779343474 @default.
- W4378976162 hasConceptScore W4378976162C2780451532 @default.
- W4378976162 hasConceptScore W4378976162C41008148 @default.
- W4378976162 hasConceptScore W4378976162C41895202 @default.
- W4378976162 hasConceptScore W4378976162C554936623 @default.
- W4378976162 hasConceptScore W4378976162C70437156 @default.
- W4378976162 hasConceptScore W4378976162C71139939 @default.
- W4378976162 hasConceptScore W4378976162C86803240 @default.
- W4378976162 hasFunder F4320321001 @default.
- W4378976162 hasFunder F4320335787 @default.
- W4378976162 hasLocation W43789761621 @default.
- W4378976162 hasOpenAccess W4378976162 @default.
- W4378976162 hasPrimaryLocation W43789761621 @default.
- W4378976162 hasRelatedWork W2081647779 @default.
- W4378976162 hasRelatedWork W2368651715 @default.
- W4378976162 hasRelatedWork W2611614995 @default.
- W4378976162 hasRelatedWork W2789919619 @default.
- W4378976162 hasRelatedWork W2792080776 @default.
- W4378976162 hasRelatedWork W3003425109 @default.
- W4378976162 hasRelatedWork W3107474891 @default.
- W4378976162 hasRelatedWork W3185852197 @default.
- W4378976162 hasRelatedWork W32283444 @default.
- W4378976162 hasRelatedWork W4321496520 @default.
- W4378976162 hasVolume "31" @default.
- W4378976162 isParatext "false" @default.
- W4378976162 isRetracted "false" @default.
- W4378976162 workType "article" @default.