Real-Time Multimodal Feedback with the CPR Tutor

D. Di Mitri*, J. Schneider Barnes, Kevin Trebing, Sasa Sopka, M.M. Specht, H.J. Drachsler

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference Article in proceedingAcademicpeer-review


We developed the CPR Tutor, a real-time multimodal feedback system for cardiopulmonary resuscitation (CPR) training. The CPR Tutor detects mistakes using recurrent neural networks for real-time time-series classification. From a multimodal data stream consisting of kinematic and electromyographic data, the CPR Tutor system automatically detects the chest compressions, which are then classified and assessed according to five performance indicators. Based on this assessment, the CPR Tutor provides audio feedback to correct the most critical mistakes and improve the CPR performance. To test the validity of the CPR Tutor, we first collected the data corpus from 10 experts used for model training. Hence, to test the impact of the feedback functionality, we ran a user study involving 10 participants. The CPR Tutor pushes forward the current state of the art of real-time multimodal tutors by providing: 1) an architecture design, 2) a methodological approach to design multimodal feedback and 3) a field study on real-time feedback for CPR training.
Original languageEnglish
Title of host publicationArtificial Intelligence in Education
Subtitle of host publication21st International Conference, AIED 2020, Ifrane, Morocco, July 6–10, 2020, Proceedings, Part I
EditorsIg Ibert Bittencourt, Mutlu Cukurova, Kasia Muldner, Rose Luckin, Eva Millán
Place of PublicationCham
PublisherSpringer International Publishing
Number of pages12
ISBN (Electronic)9783030522377
ISBN (Print)9783030522360
Publication statusPublished - 30 Jun 2020
Event21st International Conference on Artificial Intelligence in Education - Online, Unknown
Duration: 6 Jul 202010 Jul 2020
Conference number: 21

Publication series

SeriesLecture Notes in Computer Science (LNCS) series
SeriesLecture Notes in Artificial Intelligence (subseries)


Conference21st International Conference on Artificial Intelligence in Education
Abbreviated titleAIED2020


  • AI in education
  • artificial intelligence
  • cpr tutor
  • learning analytics
  • multimodal data
  • real-time feedback
  • training


Dive into the research topics of 'Real-Time Multimodal Feedback with the CPR Tutor'. Together they form a unique fingerprint.

Cite this