Communication skills training exploiting multimodal emotion recognition

Kiavash Bahreini, Rob Nadolski, Wim Westera

    Research output: Contribution to journalArticleAcademicpeer-review

    27 Downloads (Pure)


    The teaching of communication skills is a labour-intensive task because of the detailed feedback that should be given to learners during their prolonged practice. This study investigates to what extent our FILTWAM facial and vocal emotion recognition software can be used for improving a serious game (the Communication Advisor) that delivers a web-based training of communication skills. A test group of 25 participants played the game wherein they were requested to mimic specific facial and vocal emotions. Half of the assignments included direct feedback and the other half included no feedback. It was investigated whether feedback on the mimicked emotions would lead to better learning. The results suggest the facial performance growth was found to be positive, particularly significant in the feedback condition. The vocal performance growth was significant in both conditions. The results are a significant indication that the automated feedback from the software improves learners’ communication performances.
    Original languageEnglish
    Pages (from-to)1065-1082
    JournalInteractive LearnIng Environments
    Issue number8
    Early online date27 Oct 2016
    Publication statusPublished - 2017


    • Assessment of learners
    • communication skills
    • emotion recognition
    • serious gaming
    • feedback


    Dive into the research topics of 'Communication skills training exploiting multimodal emotion recognition'. Together they form a unique fingerprint.

    Cite this