Communication skills training exploiting multimodal emotion recognition

Kiavash Bahreini, Rob Nadolski, Wim Westera

Research output: Contribution to journalArticleAcademicpeer-review

2 Downloads (Pure)

Abstract

The teaching of communication skills is a labour-intensive task because of the detailed feedback that should be given to learners during their prolonged practice. This study investigates to what extent our FILTWAM facial and vocal emotion recognition software can be used for improving a serious game (the Communication Advisor) that delivers a web-based training of communication skills. A test group of 25 participants played the game wherein they were requested to mimic specific facial and vocal emotions. Half of the assignments included direct feedback and the other half included no feedback. It was investigated whether feedback on the mimicked emotions would lead to better learning. The results suggest the facial performance growth was found to be positive, particularly significant in the feedback condition. The vocal performance growth was significant in both conditions. The results are a significant indication that the automated feedback from the software improves learners’ communication performances.
Original languageEnglish
Pages (from-to)1065-1082
JournalInteractive LearnIng Environments
Volume25
Issue number8
Early online date27 Oct 2016
DOIs
Publication statusPublished - 2017

Fingerprint

communication skills
emotion
Feedback
Communication
performance
communication
indication
Teaching
Personnel
labor
learning
Group

Keywords

  • Assessment of learners
  • communication skills
  • emotion recognition
  • serious gaming
  • feedback

Cite this

@article{43592fb4ebc84c4e8b7feb2d37f0e180,
title = "Communication skills training exploiting multimodal emotion recognition",
abstract = "The teaching of communication skills is a labour-intensive task because of the detailed feedback that should be given to learners during their prolonged practice. This study investigates to what extent our FILTWAM facial and vocal emotion recognition software can be used for improving a serious game (the Communication Advisor) that delivers a web-based training of communication skills. A test group of 25 participants played the game wherein they were requested to mimic specific facial and vocal emotions. Half of the assignments included direct feedback and the other half included no feedback. It was investigated whether feedback on the mimicked emotions would lead to better learning. The results suggest the facial performance growth was found to be positive, particularly significant in the feedback condition. The vocal performance growth was significant in both conditions. The results are a significant indication that the automated feedback from the software improves learners’ communication performances.",
keywords = "Assessment of learners, communication skills, emotion recognition, serious gaming, feedback",
author = "Kiavash Bahreini and Rob Nadolski and Wim Westera",
year = "2017",
doi = "10.1080/10494820.2016.1247286",
language = "English",
volume = "25",
pages = "1065--1082",
journal = "Interactive LearnIng Environments",
issn = "1049-4820",
publisher = "Routledge",
number = "8",

}

Communication skills training exploiting multimodal emotion recognition. / Bahreini, Kiavash; Nadolski, Rob; Westera, Wim.

In: Interactive LearnIng Environments, Vol. 25, No. 8, 2017, p. 1065-1082.

Research output: Contribution to journalArticleAcademicpeer-review

TY - JOUR

T1 - Communication skills training exploiting multimodal emotion recognition

AU - Bahreini, Kiavash

AU - Nadolski, Rob

AU - Westera, Wim

PY - 2017

Y1 - 2017

N2 - The teaching of communication skills is a labour-intensive task because of the detailed feedback that should be given to learners during their prolonged practice. This study investigates to what extent our FILTWAM facial and vocal emotion recognition software can be used for improving a serious game (the Communication Advisor) that delivers a web-based training of communication skills. A test group of 25 participants played the game wherein they were requested to mimic specific facial and vocal emotions. Half of the assignments included direct feedback and the other half included no feedback. It was investigated whether feedback on the mimicked emotions would lead to better learning. The results suggest the facial performance growth was found to be positive, particularly significant in the feedback condition. The vocal performance growth was significant in both conditions. The results are a significant indication that the automated feedback from the software improves learners’ communication performances.

AB - The teaching of communication skills is a labour-intensive task because of the detailed feedback that should be given to learners during their prolonged practice. This study investigates to what extent our FILTWAM facial and vocal emotion recognition software can be used for improving a serious game (the Communication Advisor) that delivers a web-based training of communication skills. A test group of 25 participants played the game wherein they were requested to mimic specific facial and vocal emotions. Half of the assignments included direct feedback and the other half included no feedback. It was investigated whether feedback on the mimicked emotions would lead to better learning. The results suggest the facial performance growth was found to be positive, particularly significant in the feedback condition. The vocal performance growth was significant in both conditions. The results are a significant indication that the automated feedback from the software improves learners’ communication performances.

KW - Assessment of learners

KW - communication skills

KW - emotion recognition

KW - serious gaming

KW - feedback

U2 - 10.1080/10494820.2016.1247286

DO - 10.1080/10494820.2016.1247286

M3 - Article

VL - 25

SP - 1065

EP - 1082

JO - Interactive LearnIng Environments

JF - Interactive LearnIng Environments

SN - 1049-4820

IS - 8

ER -