Multimodal Emotion Recognition for Assessment of Learning in a Game-Based Communication Skills Training

Kiavash Bahreini, Rob Nadolski, Wim Westera

    Research output: Chapter in Book/Report/Conference proceedingConference Article in proceedingAcademicpeer-review

    16 Downloads (Pure)


    This paper describes how our FILTWAM software artifacts for face and voice emotion recognition will be used for assessing learners' progress and providing adequate feedback in an online game-based communication skills training. This constitutes an example of in-game assessment for mainly formative purposes. During this training, learners are requested to mimic specific emotions via a webcam and a microphone in which the software artifacts determine the adequacy of the mimicked emotion from either face and/or voice. Our previous studies have shown that these software artifacts are able to detect face and voice emotions in real-time and with sufficient reliability. In our current work, we present a software system architecture that unobtrusively monitors learners’ behaviors in an online game- based approach and offers timely and relevant feedback based upon learner’s face and voice expressions. Whereas emotion detection is often used for adapting learning content or learning tasks, our approach focuses on using emotions for guiding learners towards improved communication skills. Herein, learners need to have an opportunity of frequent guided practice in order to learn how to express the right emotion at the right time. We assume that this approach can address several issues with the current trainings in this area. We sketch the research design of our planned study that investigates the efficiency, effectiveness and enjoyableness of our approach. We conclude the paper by considering the challenges of this study.
    Original languageEnglish
    Title of host publicationLearning Analytics For and In Serious Games
    Subtitle of host publicationJoint Workshop of the GALA Network of Excellence and the LEA's BOX Project at EC-TEL 2014
    Place of PublicationGraz
    PublisherGALA and LEA's BOX projects
    Publication statusPublished - 17 Sept 2014
    Event9th European Conference on Technology Enhanced Learning: Open Learning and Teaching in Educational Communities - Graz, Austria
    Duration: 16 Sept 201419 Sept 2014
    Conference number: 9


    Conference9th European Conference on Technology Enhanced Learning
    Abbreviated titleEC-TEL 2014
    Internet address


    • Formative assessment
    • Communication skills
    • Multimodal emotion recognition
    • Serious gaming
    • Software development
    • Feedback provision


    Dive into the research topics of 'Multimodal Emotion Recognition for Assessment of Learning in a Game-Based Communication Skills Training'. Together they form a unique fingerprint.

    Cite this