Matches in SemOpenAlex for { <https://semopenalex.org/work/W3171622771> ?p ?o ?g. }
- W3171622771 abstract "In this paper, we present a video-based learning framework for animating personalized 3D talking faces from audio. We introduce two training-time data normalizations that significantly improve data sample efficiency. First, we isolate and represent faces in a normalized space that decouples 3D geometry, head pose, and texture. This decomposes the prediction problem into regressions over the 3D face shape and the corresponding 2D texture atlas. Second, we leverage facial symmetry and approximate albedo constancy of skin to isolate and remove spatio-temporal lighting variations. Together, these normalizations allow simple networks to generate high fidelity lip-sync videos under novel ambient illumination while training with just a single speaker-specific video. Further, to stabilize temporal dynamics, we introduce an auto-regressive approach that conditions the model on its previous visual state. Human ratings and objective metrics demonstrate that our method outperforms contemporary state-of-the-art audio-driven video reenactment benchmarks in terms of realism, lip-sync and visual quality scores. We illustrate several applications enabled by our framework." @default.
- W3171622771 created "2021-06-22" @default.
- W3171622771 creator A5045503768 @default.
- W3171622771 creator A5055292307 @default.
- W3171622771 creator A5066043925 @default.
- W3171622771 creator A5080976060 @default.
- W3171622771 creator A5082039339 @default.
- W3171622771 date "2021-06-08" @default.
- W3171622771 modified "2023-09-27" @default.
- W3171622771 title "LipSync3D: Data-Efficient Learning of Personalized 3D Talking Faces from Video using Pose and Lighting Normalization" @default.
- W3171622771 cites W1922655562 @default.
- W3171622771 cites W2015143272 @default.
- W3171622771 cites W2016589492 @default.
- W3171622771 cites W2029199293 @default.
- W3171622771 cites W2034129799 @default.
- W3171622771 cites W2079686637 @default.
- W3171622771 cites W2092686894 @default.
- W3171622771 cites W2113713975 @default.
- W3171622771 cites W2128019145 @default.
- W3171622771 cites W2147025387 @default.
- W3171622771 cites W2147885303 @default.
- W3171622771 cites W2162220380 @default.
- W3171622771 cites W2301937176 @default.
- W3171622771 cites W2468212864 @default.
- W3171622771 cites W2578229578 @default.
- W3171622771 cites W2604379605 @default.
- W3171622771 cites W2738406145 @default.
- W3171622771 cites W2739192055 @default.
- W3171622771 cites W2769666294 @default.
- W3171622771 cites W2806833697 @default.
- W3171622771 cites W28633752 @default.
- W3171622771 cites W2884460600 @default.
- W3171622771 cites W2906447902 @default.
- W3171622771 cites W2914037665 @default.
- W3171622771 cites W2944294033 @default.
- W3171622771 cites W2950448185 @default.
- W3171622771 cites W2959650063 @default.
- W3171622771 cites W2960274051 @default.
- W3171622771 cites W2960529565 @default.
- W3171622771 cites W2963009026 @default.
- W3171622771 cites W2963081548 @default.
- W3171622771 cites W2963125871 @default.
- W3171622771 cites W2963290645 @default.
- W3171622771 cites W2963568578 @default.
- W3171622771 cites W2981263323 @default.
- W3171622771 cites W2999966482 @default.
- W3171622771 cites W3025521275 @default.
- W3171622771 cites W3094612121 @default.
- W3171622771 cites W3097792222 @default.
- W3171622771 cites W3105763085 @default.
- W3171622771 cites W788203010 @default.
- W3171622771 hasPublicationYear "2021" @default.
- W3171622771 type Work @default.
- W3171622771 sameAs 3171622771 @default.
- W3171622771 citedByCount "0" @default.
- W3171622771 crossrefType "posted-content" @default.
- W3171622771 hasAuthorship W3171622771A5045503768 @default.
- W3171622771 hasAuthorship W3171622771A5055292307 @default.
- W3171622771 hasAuthorship W3171622771A5066043925 @default.
- W3171622771 hasAuthorship W3171622771A5080976060 @default.
- W3171622771 hasAuthorship W3171622771A5082039339 @default.
- W3171622771 hasConcept C136886441 @default.
- W3171622771 hasConcept C144024400 @default.
- W3171622771 hasConcept C153083717 @default.
- W3171622771 hasConcept C154945302 @default.
- W3171622771 hasConcept C19165224 @default.
- W3171622771 hasConcept C31972630 @default.
- W3171622771 hasConcept C41008148 @default.
- W3171622771 hasConceptScore W3171622771C136886441 @default.
- W3171622771 hasConceptScore W3171622771C144024400 @default.
- W3171622771 hasConceptScore W3171622771C153083717 @default.
- W3171622771 hasConceptScore W3171622771C154945302 @default.
- W3171622771 hasConceptScore W3171622771C19165224 @default.
- W3171622771 hasConceptScore W3171622771C31972630 @default.
- W3171622771 hasConceptScore W3171622771C41008148 @default.
- W3171622771 hasLocation W31716227711 @default.
- W3171622771 hasOpenAccess W3171622771 @default.
- W3171622771 hasPrimaryLocation W31716227711 @default.
- W3171622771 hasRelatedWork W155684729 @default.
- W3171622771 hasRelatedWork W1572174290 @default.
- W3171622771 hasRelatedWork W1930799357 @default.
- W3171622771 hasRelatedWork W2012048984 @default.
- W3171622771 hasRelatedWork W2605323213 @default.
- W3171622771 hasRelatedWork W268331225 @default.
- W3171622771 hasRelatedWork W2798896170 @default.
- W3171622771 hasRelatedWork W2799055999 @default.
- W3171622771 hasRelatedWork W2894023690 @default.
- W3171622771 hasRelatedWork W2904090037 @default.
- W3171622771 hasRelatedWork W2944294033 @default.
- W3171622771 hasRelatedWork W2972488509 @default.
- W3171622771 hasRelatedWork W2980455948 @default.
- W3171622771 hasRelatedWork W2981514602 @default.
- W3171622771 hasRelatedWork W3031133827 @default.
- W3171622771 hasRelatedWork W3048510980 @default.
- W3171622771 hasRelatedWork W3162071730 @default.
- W3171622771 hasRelatedWork W3174863185 @default.
- W3171622771 hasRelatedWork W3175779516 @default.
- W3171622771 hasRelatedWork W3182700213 @default.
- W3171622771 isParatext "false" @default.
- W3171622771 isRetracted "false" @default.