MUHCI – “Multi-modal Human-Computer Interaction”

Overview

MUHCI focuses on human-centered interfaces. Our approach is to develop a human-like computer that can understand the behavior of human beings, can express its emotions to us, and can communicate and interact with us in natural ways. Like a normal human, such a smart computer should have functional blocks: sensors, brain, and actuators. In the project, a number of challenging, closely coupled analysis tasks is identified, namely, (a) robustifying face analysis (face localisation, facial feature extraction, facial feature tracking); (b) face verification; (c) facial expression and face gesture recognition; (d) hand gesture recognition; (e) automatic interpretation of user’s behavior from cursor movements; (f) audio-visual speech recognition; (g) audio-visual speech synthesis; (h) improving 3-D face modeling; (i) facial animation providing higher quality of emotions. The aforementioned analysis tasks correspond to the “sensors+brain” part.

Synthesis tasks, such as the development of talking heads and active browsers, are addressed, that correspond to the “brain+actuators” part. Other applications, closely related to the above defined tasks, are also promoted, i.e., (a) developing, extending and improving several techniques for building databases of faces and bodies (cloning from photographs, morphing, generating populations, manual editing), and publishing facial animation content on the Internet; (b) model-based natural-synthetic coding.

Human-computer bi-directional interaction needs a mixture of experts coming from different disciplines, such as, image processing, computer vision, speech processing, pattern recognition, computer graphics, communications, linguistics, etc. It provides a fertile ground for exchanging ideas and cooperation that will greatly facilitate the proper treatment of the aforementioned complicated problems. A work plan is set up to provide solutions that improve the state-of-the-art techniques used at each problem. The quality of the solutions that will be given to each research topic depends strongly on the synergy and the cooperation of experts from different fields.

Partners

Aristotle University of Thessaloniki
Greece
www.auth.gr

Helsinki University of Technology
Finland
www.lce.hut.fi

Linköpings universitet
Sweden
www.isy.liu.se/en

MIRALab, University of Geneva
Switzerland
www.miralab.ch

Ruhr-Universitaet Bochum
Germany
www.neuroinformatik.ruhr-uni-bochum.de

Tampere University of Technology
Finland
www.tut.fi

Umea University
Sweden
www.umu.se

W Interactive SARL
France
www.winteractive.fr