FILTWAM and Voice Emotion Recognition

Kiavash Bahreini*, Rob Nadolski, Wim Westera

*Corresponding author for this work

    Research output: Chapter in Book/Report/Conference proceedingConference Article in proceedingAcademicpeer-review

    1 Downloads (Pure)

    Abstract

    This paper introduces the voice emotion recognition part of our framework for improving learning through webcams and microphones (FILTWAM). This framework enables multimodal emotion recognition of learners during game-based learning. The main goal of this study is to validate the use of microphone data for a real-time and adequate interpretation of vocal expressions into emotional states were the software is calibrated with end users. FILTWAM already incorporates a valid face emotion recognition module and is extended with a voice emotion recognition module. This extension aims to provide relevant and timely feedback based upon learner's vocal intonations. The feedback is expected to enhance learner’s awareness of his or her own behavior. Six test persons received the same computer-based tasks in which they were requested to mimic specific vocal expressions. Each test person mimicked 82 emotions, which led to a dataset of 492 emotions. All sessions were recorded on video. An overall accuracy of our software based on the requested emotions and the recognized emotions is a pretty good 74.6% for the emotions happy and neutral emotions; but will be improved for the lower values of an extended set of emotions. In contrast with existing software our solution allows to continuously and unobtrusively monitor learners’ intonations and convert these intonations into emotional states. This paves the way for enhancing the quality and efficacy of game-based learning by including the learner's emotional states, and links these to pedagogical scaffolding.
    Original languageEnglish
    Title of host publicationGames and Learning Alliance.
    Subtitle of host publicationGALA 2013
    EditorsA. De Gloria
    Place of PublicationCham
    PublisherSpringer
    Pages116-129
    Number of pages14
    ISBN (Electronic)978-3-319-12157-4
    ISBN (Print)978-3-319-12156-7
    DOIs
    Publication statusPublished - 26 Oct 2014
    Event2th International Conference Games and Learning Alliance - Paris, France
    Duration: 23 Oct 201325 Oct 2013
    Conference number: 2

    Publication series

    SeriesLecture Notes in Computer Science
    Volume8605
    ISSN1611-3349

    Conference

    Conference2th International Conference Games and Learning Alliance
    Abbreviated titleGALA 2013
    Country/TerritoryFrance
    CityParis
    Period23/10/1325/10/13

    Keywords

    • Learner support in serious games
    • game-based learning
    • human-computer interaction
    • multimodal emotion recognition
    • real-time voice emotion recognition
    • microphone

    Fingerprint

    Dive into the research topics of 'FILTWAM and Voice Emotion Recognition'. Together they form a unique fingerprint.

    Cite this