Matches in SemOpenAlex for { <https://semopenalex.org/work/W4386875504> ?p ?o ?g. }
Showing items 1 to 63 of
63
with 100 items per page.
- W4386875504 abstract "Gestures are non-verbal but important behaviors accompanying people's speech. While previous methods are able to generate speech rhythm-synchronized gestures, the semantic context of the speech is generally lacking in the gesticulations. Although semantic gestures do not occur very regularly in human speech, they are indeed the key for the audience to understand the speech context in a more immersive environment. Hence, we introduce LivelySpeaker, a framework that realizes semantics-aware co-speech gesture generation and offers several control handles. In particular, our method decouples the task into two stages: script-based gesture generation and audio-guided rhythm refinement. Specifically, the script-based gesture generation leverages the pre-trained CLIP text embeddings as the guidance for generating gestures that are highly semantically aligned with the script. Then, we devise a simple but effective diffusion-based gesture generation backbone simply using pure MLPs, that is conditioned on only audio signals and learns to gesticulate with realistic motions. We utilize such powerful prior to rhyme the script-guided gestures with the audio signals, notably in a zero-shot setting. Our novel two-stage generation framework also enables several applications, such as changing the gesticulation style, editing the co-speech gestures via textual prompting, and controlling the semantic awareness and rhythm alignment with guided diffusion. Extensive experiments demonstrate the advantages of the proposed framework over competing methods. In addition, our core diffusion-based generative model also achieves state-of-the-art performance on two benchmarks. The code and model will be released to facilitate future research." @default.
- W4386875504 created "2023-09-20" @default.
- W4386875504 creator A5009598331 @default.
- W4386875504 creator A5014268203 @default.
- W4386875504 creator A5017818991 @default.
- W4386875504 creator A5023310229 @default.
- W4386875504 creator A5034339267 @default.
- W4386875504 creator A5058799911 @default.
- W4386875504 creator A5087161777 @default.
- W4386875504 date "2023-09-17" @default.
- W4386875504 modified "2023-09-26" @default.
- W4386875504 title "LivelySpeaker: Towards Semantic-Aware Co-Speech Gesture Generation" @default.
- W4386875504 doi "https://doi.org/10.48550/arxiv.2309.09294" @default.
- W4386875504 hasPublicationYear "2023" @default.
- W4386875504 type Work @default.
- W4386875504 citedByCount "0" @default.
- W4386875504 crossrefType "posted-content" @default.
- W4386875504 hasAuthorship W4386875504A5009598331 @default.
- W4386875504 hasAuthorship W4386875504A5014268203 @default.
- W4386875504 hasAuthorship W4386875504A5017818991 @default.
- W4386875504 hasAuthorship W4386875504A5023310229 @default.
- W4386875504 hasAuthorship W4386875504A5034339267 @default.
- W4386875504 hasAuthorship W4386875504A5058799911 @default.
- W4386875504 hasAuthorship W4386875504A5087161777 @default.
- W4386875504 hasBestOaLocation W43868755041 @default.
- W4386875504 hasConcept C151730666 @default.
- W4386875504 hasConcept C154945302 @default.
- W4386875504 hasConcept C159437735 @default.
- W4386875504 hasConcept C184337299 @default.
- W4386875504 hasConcept C199360897 @default.
- W4386875504 hasConcept C204321447 @default.
- W4386875504 hasConcept C207347870 @default.
- W4386875504 hasConcept C2779343474 @default.
- W4386875504 hasConcept C28490314 @default.
- W4386875504 hasConcept C41008148 @default.
- W4386875504 hasConcept C86803240 @default.
- W4386875504 hasConceptScore W4386875504C151730666 @default.
- W4386875504 hasConceptScore W4386875504C154945302 @default.
- W4386875504 hasConceptScore W4386875504C159437735 @default.
- W4386875504 hasConceptScore W4386875504C184337299 @default.
- W4386875504 hasConceptScore W4386875504C199360897 @default.
- W4386875504 hasConceptScore W4386875504C204321447 @default.
- W4386875504 hasConceptScore W4386875504C207347870 @default.
- W4386875504 hasConceptScore W4386875504C2779343474 @default.
- W4386875504 hasConceptScore W4386875504C28490314 @default.
- W4386875504 hasConceptScore W4386875504C41008148 @default.
- W4386875504 hasConceptScore W4386875504C86803240 @default.
- W4386875504 hasLocation W43868755041 @default.
- W4386875504 hasOpenAccess W4386875504 @default.
- W4386875504 hasPrimaryLocation W43868755041 @default.
- W4386875504 hasRelatedWork W1974379374 @default.
- W4386875504 hasRelatedWork W1999635775 @default.
- W4386875504 hasRelatedWork W2111894689 @default.
- W4386875504 hasRelatedWork W2140121947 @default.
- W4386875504 hasRelatedWork W2945648453 @default.
- W4386875504 hasRelatedWork W2984615118 @default.
- W4386875504 hasRelatedWork W4281626041 @default.
- W4386875504 hasRelatedWork W4316659390 @default.
- W4386875504 hasRelatedWork W2005997082 @default.
- W4386875504 hasRelatedWork W2520877275 @default.
- W4386875504 isParatext "false" @default.
- W4386875504 isRetracted "false" @default.
- W4386875504 workType "article" @default.