Detecting Mistakes in CPR Training with Multimodal Data and Neural Networks

Daniele Di Mitri*, Jan Schneider, Marcus Specht, Hendrik Drachsler

*Corresponding author for this work

Research output: Contribution to journal › Article › Academic › peer-review

Abstract

This study investigated to what extent multimodal data can be used to detect mistakesduring Cardiopulmonary Resuscitation (CPR) training. We complemented the Laerdal QCPRResusciAnne manikin with the Multimodal Tutor for CPR, a multi-sensor system consisting ofa Microsoft Kinect for tracking body position and a Myo armband for collecting electromyograminformation. We collected multimodal data from 11 medical students, each of them performing twosessions of two-minute chest compressions (CCs). We gathered in total 5254 CCs that were all labelledaccording to five performance indicators, corresponding to common CPR training mistakes. Three outof five indicators, CC rate, CC depth and CC release, were assessed automatically by the ReusciAnnemanikin. The remaining two, related to arms and body position, were annotated manually by theresearch team. We trained five neural networks for classifying each of the five indicators. The resultsof the experiment show that multimodal data can provide accurate mistake detection as comparedto the ResusciAnne manikin baseline. We also show that the Multimodal Tutor for CPR can detectadditional CPR training mistakes such as the correct use of arms and body weight. Thus far, thesemistakes were identified only by human instructors. Finally, to investigate user feedback in the futureimplementations of the Multimodal Tutor for CPR, we conducted a questionnaire to collect valuablefeedback aspects of CPR training.
Original languageEnglish
Article number3099
Number of pages20
JournalSensors
Volume19
Issue number14
DOIs
Publication statusPublished - 13 Jul 2019

Fingerprint

resuscitation
Resuscitation
Cardiopulmonary Resuscitation
chest
education
Neural networks
Thorax
Manikins
Arm
body weight
Sensor data fusion
instructors
classifying
Medical Students
students
Body Weight
Students
Feedback

Keywords

  • activity recognition
  • learning analytics
  • medical simulation
  • multimodal data
  • neural networks
  • psychomotor learning
  • sensors
  • signal processing
  • training mistakes

Cite this

Di Mitri, Daniele ; Schneider, Jan ; Specht, Marcus ; Drachsler, Hendrik. / Detecting Mistakes in CPR Training with Multimodal Data and Neural Networks. In: Sensors. 2019 ; Vol. 19, No. 14.
@article{43c147b7ad734623837b660171083809,
title = "Detecting Mistakes in CPR Training with Multimodal Data and Neural Networks",
abstract = "This study investigated to what extent multimodal data can be used to detect mistakesduring Cardiopulmonary Resuscitation (CPR) training. We complemented the Laerdal QCPRResusciAnne manikin with the Multimodal Tutor for CPR, a multi-sensor system consisting ofa Microsoft Kinect for tracking body position and a Myo armband for collecting electromyograminformation. We collected multimodal data from 11 medical students, each of them performing twosessions of two-minute chest compressions (CCs). We gathered in total 5254 CCs that were all labelledaccording to five performance indicators, corresponding to common CPR training mistakes. Three outof five indicators, CC rate, CC depth and CC release, were assessed automatically by the ReusciAnnemanikin. The remaining two, related to arms and body position, were annotated manually by theresearch team. We trained five neural networks for classifying each of the five indicators. The resultsof the experiment show that multimodal data can provide accurate mistake detection as comparedto the ResusciAnne manikin baseline. We also show that the Multimodal Tutor for CPR can detectadditional CPR training mistakes such as the correct use of arms and body weight. Thus far, thesemistakes were identified only by human instructors. Finally, to investigate user feedback in the futureimplementations of the Multimodal Tutor for CPR, we conducted a questionnaire to collect valuablefeedback aspects of CPR training.",
keywords = "activity recognition, learning analytics, medical simulation, multimodal data, neural networks, psychomotor learning, sensors, signal processing, training mistakes",
author = "{Di Mitri}, Daniele and Jan Schneider and Marcus Specht and Hendrik Drachsler",
year = "2019",
month = "7",
day = "13",
doi = "10.3390/s19143099",
language = "English",
volume = "19",
journal = "Sensors",
issn = "1424-8220",
publisher = "MDPI",
number = "14",

}

Di Mitri, D, Schneider, J, Specht, M & Drachsler, H 2019, 'Detecting Mistakes in CPR Training with Multimodal Data and Neural Networks', Sensors, vol. 19, no. 14, 3099. https://doi.org/10.3390/s19143099

Detecting Mistakes in CPR Training with Multimodal Data and Neural Networks. / Di Mitri, Daniele; Schneider, Jan; Specht, Marcus; Drachsler, Hendrik.

In: Sensors, Vol. 19, No. 14, 3099, 13.07.2019.

Research output: Contribution to journal › Article › Academic › peer-review

TY - JOUR

T1 - Detecting Mistakes in CPR Training with Multimodal Data and Neural Networks

AU - Di Mitri, Daniele

AU - Schneider, Jan

AU - Specht, Marcus

AU - Drachsler, Hendrik

PY - 2019/7/13

Y1 - 2019/7/13

N2 - This study investigated to what extent multimodal data can be used to detect mistakesduring Cardiopulmonary Resuscitation (CPR) training. We complemented the Laerdal QCPRResusciAnne manikin with the Multimodal Tutor for CPR, a multi-sensor system consisting ofa Microsoft Kinect for tracking body position and a Myo armband for collecting electromyograminformation. We collected multimodal data from 11 medical students, each of them performing twosessions of two-minute chest compressions (CCs). We gathered in total 5254 CCs that were all labelledaccording to five performance indicators, corresponding to common CPR training mistakes. Three outof five indicators, CC rate, CC depth and CC release, were assessed automatically by the ReusciAnnemanikin. The remaining two, related to arms and body position, were annotated manually by theresearch team. We trained five neural networks for classifying each of the five indicators. The resultsof the experiment show that multimodal data can provide accurate mistake detection as comparedto the ResusciAnne manikin baseline. We also show that the Multimodal Tutor for CPR can detectadditional CPR training mistakes such as the correct use of arms and body weight. Thus far, thesemistakes were identified only by human instructors. Finally, to investigate user feedback in the futureimplementations of the Multimodal Tutor for CPR, we conducted a questionnaire to collect valuablefeedback aspects of CPR training.

AB - This study investigated to what extent multimodal data can be used to detect mistakesduring Cardiopulmonary Resuscitation (CPR) training. We complemented the Laerdal QCPRResusciAnne manikin with the Multimodal Tutor for CPR, a multi-sensor system consisting ofa Microsoft Kinect for tracking body position and a Myo armband for collecting electromyograminformation. We collected multimodal data from 11 medical students, each of them performing twosessions of two-minute chest compressions (CCs). We gathered in total 5254 CCs that were all labelledaccording to five performance indicators, corresponding to common CPR training mistakes. Three outof five indicators, CC rate, CC depth and CC release, were assessed automatically by the ReusciAnnemanikin. The remaining two, related to arms and body position, were annotated manually by theresearch team. We trained five neural networks for classifying each of the five indicators. The resultsof the experiment show that multimodal data can provide accurate mistake detection as comparedto the ResusciAnne manikin baseline. We also show that the Multimodal Tutor for CPR can detectadditional CPR training mistakes such as the correct use of arms and body weight. Thus far, thesemistakes were identified only by human instructors. Finally, to investigate user feedback in the futureimplementations of the Multimodal Tutor for CPR, we conducted a questionnaire to collect valuablefeedback aspects of CPR training.

KW - activity recognition

KW - learning analytics

KW - medical simulation

KW - multimodal data

KW - neural networks

KW - psychomotor learning

KW - sensors

KW - signal processing

KW - training mistakes

U2 - 10.3390/s19143099

DO - 10.3390/s19143099

M3 - Article

C2 - 31337029

VL - 19

JO - Sensors

JF - Sensors

SN - 1424-8220

IS - 14

M1 - 3099

ER -

Di Mitri D, Schneider J, Specht M, Drachsler H. Detecting Mistakes in CPR Training with Multimodal Data and Neural Networks. Sensors. 2019 Jul 13;19(14). 3099. https://doi.org/10.3390/s19143099

Access to Document