Detecting Mistakes in CPR Training with Multimodal Data and Neural Networks

Daniele Di Mitri*, Jan Schneider, Marcus Specht, Hendrik Drachsler

*Corresponding author for this work

    Research output: Contribution to journalArticleAcademicpeer-review


    This study investigated to what extent multimodal data can be used to detect mistakesduring Cardiopulmonary Resuscitation (CPR) training. We complemented the Laerdal QCPRResusciAnne manikin with the Multimodal Tutor for CPR, a multi-sensor system consisting ofa Microsoft Kinect for tracking body position and a Myo armband for collecting electromyograminformation. We collected multimodal data from 11 medical students, each of them performing twosessions of two-minute chest compressions (CCs). We gathered in total 5254 CCs that were all labelledaccording to five performance indicators, corresponding to common CPR training mistakes. Three outof five indicators, CC rate, CC depth and CC release, were assessed automatically by the ReusciAnnemanikin. The remaining two, related to arms and body position, were annotated manually by theresearch team. We trained five neural networks for classifying each of the five indicators. The resultsof the experiment show that multimodal data can provide accurate mistake detection as comparedto the ResusciAnne manikin baseline. We also show that the Multimodal Tutor for CPR can detectadditional CPR training mistakes such as the correct use of arms and body weight. Thus far, thesemistakes were identified only by human instructors. Finally, to investigate user feedback in the futureimplementations of the Multimodal Tutor for CPR, we conducted a questionnaire to collect valuablefeedback aspects of CPR training.
    Original languageEnglish
    Article number3099
    Number of pages20
    Issue number14
    Publication statusPublished - 13 Jul 2019


    • activity recognition
    • learning analytics
    • medical simulation
    • multimodal data
    • neural networks
    • psychomotor learning
    • sensors
    • signal processing
    • training mistakes


    Dive into the research topics of 'Detecting Mistakes in CPR Training with Multimodal Data and Neural Networks'. Together they form a unique fingerprint.

    Cite this