Prosody Driven Head-Gesture Animation

Motivation:

• State of the art visual speaker animation methods,
• capable of synchronized lip synthesis,
• lack synchronized gesture synthesis
• Head and face gestures are usually added manually by artists,
• costly and often look unrealistic
• Head gestures are correlated with speech (prosody)
• Can we learn correlation model between prosody and head-gestures?

Demo I: Head-gesture animation	Demo II: Prosody transplant

A short presentation on prosody driven head-gesture animation:



Publications:

1. M. Emre Sargın. Audio-visual correlation modeling for speaker identification and synthesis. Master’s thesis, Koc University, 2006.
   
2. M.E. Sargın, Y. Yemez, E. Erzin, and A.M. Tekalp, "Analysis of Head Gesture and Prosody Patterns for Prosody-Driven Head-Gesture Animation," IEEE Transactions on Pattern Analysis and Machine Intelligence, Volume 30, Issue 8, pp. 1330 - 1345, Agust 2008.
3. M.E. Sargın, Y. Yemez, E. Erzin, and A.M. Tekalp, "Audio-Visual Synchronization and Fusion using Canonical Correlation Analysis," IEEE Transactions on Multimedia, Vol. 9, No. 7, pp. 1396-1403, November 2007.
4. M. E. Sargin, E. Erzin, Y. Yemez, A. M. Tekalp, A. T. Erdem, C. Erdem, M. Ozkan "Prosody-Driven Head-Gesture Animation," ICASSP'07.
5. M.E. Sargin, O. Aran, A. Karpov,F. Ofli, Y. Yasinnik, S. Wilson, E. Erzin, Y. Yemez and A.M. Tekalp " Combined Gesture-Speech Analysis and Speech Driven Gesture Synthesis," IEEE Int. Conf. on Multimedia Expo, Toronto, 2006.

Funded by:

• A. M. Tekalp, E. Erzin, Y. Yemez, ``SIMILAR: The European taskforce creating human-machine interfaces SIMILAR to human-human communication,'' FP6 Network of Excellence, 2003-2007.

TOP