MIMiC: Multimodal Interactive Motion Controller

Abstract
We introduce a new algorithm for real-time interactive motion control and demonstrate its application to motion captured data, pre-recorded videos and HCI. Firstly, a data set of frames are projected into a lower dimensional space. An appearance model is learnt using a multivariate probability distribution. A novel approach to determining transition points is presented based on k-medoids, whereby appropriate points of intersection in the motion trajectory are derived as cluster centres. These points are used to segment the data into smaller subsequences. A transition matrix combined with a kernel density estimation is used to determine suitable transitions between the subsequences to develop novel motion. To facilitate real-time interactive control, conditional probabilities are used to derive motion given user commands. The user commands can come from any modality including auditory, touch and gesture. The system is also extended to HCI using audio signals of speech in a conversation to trigger non-verbal responses from a synthetic listener in real-time. We demonstrate the flexibility of the model by presenting results ranging from data sets composed of vectorised images, 2D and 3D point representations. Results show real-time interaction and plausible motion generation between different types of movement.

mocapResults8
[Image showing synthesis and blending of different types of human articulated motion captured walks]

candleall4
[Image showing candle flame synthesis]

plasmaall7
[Image showing plasma beam synthesis. Arrows indicate direction of synthesis flow]

nod2
[Image showing synthesis of nods as a response to audio stimulus]

Authors
Dumebi Okwechime
Eng-Jon Ong
Richard Bowden

Resources

iccv09_dumebi_CRC
Real-Time Motion Control Using Pose Space Probability Density Estimation
Okwechime, D. Ong, E-J. Bowden, R.
In IEEE International Workshop on Human Computer Interaction, ICCV09, Kyoto, Japan 2009.
[
pdf 1.6 MB] [bibtex]





tmmcover
MIMiC: Multimodal Interactive Motion Controller
Okwechime, D. Ong, E-J. Bowden, R.
To appear in IEEE Transactions on Multimedia, 2011
DOI: 10.1109/TMM.2010.2096410 
[
pdf 3 MB] [bibtex]





mimic2
TMM2011 Supplementary Material - Includes Audio (3:09 mins)
(Requires Quicktime to view)
[Video 6.8 MB]