Matches in SemOpenAlex for { <https://semopenalex.org/work/W4387446220> ?p ?o ?g. }
Showing items 1 to 86 of
86
with 100 items per page.
- W4387446220 abstract "Sign language is a rich form of communication, uniquely conveying meaning through a combination of gestures, facial expressions, and body movements. Existing research in sign language generation has predominantly focused on text-to-sign pose generation, while speech-to-sign pose generation remains relatively underexplored. Speech-to-sign language generation models can facilitate effective communication between the deaf and hearing communities. In this paper, we propose an architecture that utilises prosodic information from speech audio and semantic context from text to generate sign pose sequences. In our approach, we adopt a multi-tasking strategy that involves an additional task of predicting Facial Action Units (FAUs). FAUs capture the intricate facial muscle movements that play a crucial role in conveying specific facial expressions during sign language generation. We train our models on an existing Indian Sign language dataset that contains sign language videos with audio and text translations. To evaluate our models, we report Dynamic Time Warping (DTW) and Probability of Correct Keypoints (PCK) scores. We find that combining prosody and text as input, along with incorporating facial action unit prediction as an additional task, outperforms previous models in both DTW and PCK scores. We also discuss the challenges and limitations of speech-to-sign pose generation models to encourage future research in this domain. We release our models, results and code to foster reproducibility and encourage future research1." @default.
- W4387446220 created "2023-10-10" @default.
- W4387446220 creator A5013675579 @default.
- W4387446220 creator A5033269864 @default.
- W4387446220 creator A5093026439 @default.
- W4387446220 date "2023-10-09" @default.
- W4387446220 modified "2023-10-11" @default.
- W4387446220 title "MultiFacet: A Multi-Tasking Framework for Speech-to-Sign Language Generation" @default.
- W4387446220 cites W2046033161 @default.
- W4387446220 cites W2799020610 @default.
- W4387446220 cites W2972359262 @default.
- W4387446220 cites W2997573805 @default.
- W4387446220 cites W3083173864 @default.
- W4387446220 cites W3093647117 @default.
- W4387446220 cites W3098994456 @default.
- W4387446220 cites W3107972479 @default.
- W4387446220 cites W3158356393 @default.
- W4387446220 cites W3198078703 @default.
- W4387446220 cites W3198544313 @default.
- W4387446220 cites W4285253586 @default.
- W4387446220 cites W4312262410 @default.
- W4387446220 doi "https://doi.org/10.1145/3610661.3616550" @default.
- W4387446220 hasPublicationYear "2023" @default.
- W4387446220 type Work @default.
- W4387446220 citedByCount "0" @default.
- W4387446220 crossrefType "proceedings-article" @default.
- W4387446220 hasAuthorship W4387446220A5013675579 @default.
- W4387446220 hasAuthorship W4387446220A5033269864 @default.
- W4387446220 hasAuthorship W4387446220A5093026439 @default.
- W4387446220 hasBestOaLocation W43874462201 @default.
- W4387446220 hasConcept C134306372 @default.
- W4387446220 hasConcept C137293760 @default.
- W4387446220 hasConcept C138885662 @default.
- W4387446220 hasConcept C139676723 @default.
- W4387446220 hasConcept C151730666 @default.
- W4387446220 hasConcept C154945302 @default.
- W4387446220 hasConcept C162324750 @default.
- W4387446220 hasConcept C187736073 @default.
- W4387446220 hasConcept C195704467 @default.
- W4387446220 hasConcept C204321447 @default.
- W4387446220 hasConcept C207347870 @default.
- W4387446220 hasConcept C2779343474 @default.
- W4387446220 hasConcept C2780451532 @default.
- W4387446220 hasConcept C28490314 @default.
- W4387446220 hasConcept C33923547 @default.
- W4387446220 hasConcept C41008148 @default.
- W4387446220 hasConcept C41895202 @default.
- W4387446220 hasConcept C522192633 @default.
- W4387446220 hasConcept C86803240 @default.
- W4387446220 hasConcept C88516994 @default.
- W4387446220 hasConceptScore W4387446220C134306372 @default.
- W4387446220 hasConceptScore W4387446220C137293760 @default.
- W4387446220 hasConceptScore W4387446220C138885662 @default.
- W4387446220 hasConceptScore W4387446220C139676723 @default.
- W4387446220 hasConceptScore W4387446220C151730666 @default.
- W4387446220 hasConceptScore W4387446220C154945302 @default.
- W4387446220 hasConceptScore W4387446220C162324750 @default.
- W4387446220 hasConceptScore W4387446220C187736073 @default.
- W4387446220 hasConceptScore W4387446220C195704467 @default.
- W4387446220 hasConceptScore W4387446220C204321447 @default.
- W4387446220 hasConceptScore W4387446220C207347870 @default.
- W4387446220 hasConceptScore W4387446220C2779343474 @default.
- W4387446220 hasConceptScore W4387446220C2780451532 @default.
- W4387446220 hasConceptScore W4387446220C28490314 @default.
- W4387446220 hasConceptScore W4387446220C33923547 @default.
- W4387446220 hasConceptScore W4387446220C41008148 @default.
- W4387446220 hasConceptScore W4387446220C41895202 @default.
- W4387446220 hasConceptScore W4387446220C522192633 @default.
- W4387446220 hasConceptScore W4387446220C86803240 @default.
- W4387446220 hasConceptScore W4387446220C88516994 @default.
- W4387446220 hasLocation W43874462201 @default.
- W4387446220 hasOpenAccess W4387446220 @default.
- W4387446220 hasPrimaryLocation W43874462201 @default.
- W4387446220 hasRelatedWork W1986488374 @default.
- W4387446220 hasRelatedWork W2030799363 @default.
- W4387446220 hasRelatedWork W2288425735 @default.
- W4387446220 hasRelatedWork W2341338763 @default.
- W4387446220 hasRelatedWork W2349923317 @default.
- W4387446220 hasRelatedWork W2893683540 @default.
- W4387446220 hasRelatedWork W2894081631 @default.
- W4387446220 hasRelatedWork W2986063033 @default.
- W4387446220 hasRelatedWork W4206297952 @default.
- W4387446220 hasRelatedWork W4308478915 @default.
- W4387446220 isParatext "false" @default.
- W4387446220 isRetracted "false" @default.
- W4387446220 workType "article" @default.