Real-Time Multimodal Feedback with the CPR Tutor

D. Di Mitri; J. Schneider Barnes; Kevin Trebing; Sasa Sopka; M.M. Specht; H.J. Drachsler

doi:10.1007/978-3-030-52237-7_12

Real-Time Multimodal Feedback with the CPR Tutor

D. Di Mitri^*, J. Schneider Barnes, Kevin Trebing, Sasa Sopka, M.M. Specht, H.J. Drachsler

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference Article in proceeding › Academic › peer-review

Abstract

We developed the CPR Tutor, a real-time multimodal feedback system for cardiopulmonary resuscitation (CPR) training. The CPR Tutor detects mistakes using recurrent neural networks for real-time time-series classification. From a multimodal data stream consisting of kinematic and electromyographic data, the CPR Tutor system automatically detects the chest compressions, which are then classified and assessed according to five performance indicators. Based on this assessment, the CPR Tutor provides audio feedback to correct the most critical mistakes and improve the CPR performance. To test the validity of the CPR Tutor, we first collected the data corpus from 10 experts used for model training. Hence, to test the impact of the feedback functionality, we ran a user study involving 10 participants. The CPR Tutor pushes forward the current state of the art of real-time multimodal tutors by providing: 1) an architecture design, 2) a methodological approach to design multimodal feedback and 3) a field study on real-time feedback for CPR training.

Original language	English
Title of host publication	Artificial Intelligence in Education
Subtitle of host publication	21st International Conference, AIED 2020, Ifrane, Morocco, July 6–10, 2020, Proceedings, Part I
Editors	Ig Ibert Bittencourt, Mutlu Cukurova, Kasia Muldner, Rose Luckin, Eva Millán
Place of Publication	Cham
Publisher	Springer International Publishing
Chapter	12
Pages	141-152
Number of pages	12
Volume	1
ISBN (Electronic)	9783030522377
ISBN (Print)	9783030522360
DOIs	https://doi.org/10.1007/978-3-030-52237-7_12
Publication status	Published - 30 Jun 2020
Event	21st International Conference on Artificial Intelligence in Education - Online, Unknown Duration: 6 Jul 2020 → 10 Jul 2020 Conference number: 21

Publication series

Series	Lecture Notes in Computer Science (LNCS) series
Volume	12163
ISSN	0302-9743

Series	Lecture Notes in Artificial Intelligence (subseries)
Volume	12163

Conference

Conference	21st International Conference on Artificial Intelligence in Education
Abbreviated title	AIED2020
Country/Territory	Unknown
Period	6/07/20 → 10/07/20

Keywords

AI in education
artificial intelligence
cpr tutor
learning analytics
multimodal data
real-time feedback
training

Access to Document

10.1007/978-3-030-52237-7_12

Cite this

Di Mitri, D., Schneider Barnes, J., Trebing, K., Sopka, S., Specht, M. M., & Drachsler, H. J. (2020). Real-Time Multimodal Feedback with the CPR Tutor. In I. I. Bittencourt, M. Cukurova, K. Muldner, R. Luckin, & E. Millán (Eds.), Artificial Intelligence in Education: 21st International Conference, AIED 2020, Ifrane, Morocco, July 6–10, 2020, Proceedings, Part I (Vol. 1, pp. 141-152). Springer International Publishing. https://doi.org/10.1007/978-3-030-52237-7_12

Di Mitri, D. ; Schneider Barnes, J. ; Trebing, Kevin et al. / Real-Time Multimodal Feedback with the CPR Tutor. Artificial Intelligence in Education: 21st International Conference, AIED 2020, Ifrane, Morocco, July 6–10, 2020, Proceedings, Part I. editor / Ig Ibert Bittencourt ; Mutlu Cukurova ; Kasia Muldner ; Rose Luckin ; Eva Millán. Vol. 1 Cham : Springer International Publishing, 2020. pp. 141-152 (Lecture Notes in Computer Science (LNCS) series, Vol. 12163). (Lecture Notes in Artificial Intelligence (subseries), Vol. 12163).

@inproceedings{4d3e26c26af14a90904b816aad7ded68,

title = "Real-Time Multimodal Feedback with the CPR Tutor",

abstract = "We developed the CPR Tutor, a real-time multimodal feedback system for cardiopulmonary resuscitation (CPR) training. The CPR Tutor detects mistakes using recurrent neural networks for real-time time-series classification. From a multimodal data stream consisting of kinematic and electromyographic data, the CPR Tutor system automatically detects the chest compressions, which are then classified and assessed according to five performance indicators. Based on this assessment, the CPR Tutor provides audio feedback to correct the most critical mistakes and improve the CPR performance. To test the validity of the CPR Tutor, we first collected the data corpus from 10 experts used for model training. Hence, to test the impact of the feedback functionality, we ran a user study involving 10 participants. The CPR Tutor pushes forward the current state of the art of real-time multimodal tutors by providing: 1) an architecture design, 2) a methodological approach to design multimodal feedback and 3) a field study on real-time feedback for CPR training.",

keywords = "AI in education, artificial intelligence, cpr tutor, learning analytics, multimodal data, real-time feedback, training",

author = "{Di Mitri}, D. and {Schneider Barnes}, J. and Kevin Trebing and Sasa Sopka and M.M. Specht and H.J. Drachsler",

year = "2020",

month = jun,

day = "30",

doi = "10.1007/978-3-030-52237-7_12",

language = "English",

isbn = "9783030522360",

volume = "1",

series = "Lecture Notes in Computer Science (LNCS) series",

publisher = "Springer International Publishing",

pages = "141--152",

editor = "Bittencourt, {Ig Ibert} and Mutlu Cukurova and Kasia Muldner and Rose Luckin and Eva Mill{\'a}n",

booktitle = "Artificial Intelligence in Education",

address = "Switzerland",

note = "21st International Conference on Artificial Intelligence in Education, AIED2020 ; Conference date: 06-07-2020 Through 10-07-2020",

}

Di Mitri, D, Schneider Barnes, J, Trebing, K, Sopka, S, Specht, MM & Drachsler, HJ 2020, Real-Time Multimodal Feedback with the CPR Tutor. in II Bittencourt, M Cukurova, K Muldner, R Luckin & E Millán (eds), Artificial Intelligence in Education: 21st International Conference, AIED 2020, Ifrane, Morocco, July 6–10, 2020, Proceedings, Part I. vol. 1, Springer International Publishing, Cham, Lecture Notes in Computer Science (LNCS) series, vol. 12163, Lecture Notes in Artificial Intelligence (subseries), vol. 12163, pp. 141-152, 21st International Conference on Artificial Intelligence in Education, Unknown, 6/07/20. https://doi.org/10.1007/978-3-030-52237-7_12

Real-Time Multimodal Feedback with the CPR Tutor. / Di Mitri, D.; Schneider Barnes, J.; Trebing, Kevin et al.
Artificial Intelligence in Education: 21st International Conference, AIED 2020, Ifrane, Morocco, July 6–10, 2020, Proceedings, Part I. ed. / Ig Ibert Bittencourt; Mutlu Cukurova; Kasia Muldner; Rose Luckin; Eva Millán. Vol. 1 Cham: Springer International Publishing, 2020. p. 141-152 (Lecture Notes in Computer Science (LNCS) series, Vol. 12163). (Lecture Notes in Artificial Intelligence (subseries), Vol. 12163).

Research output: Chapter in Book/Report/Conference proceeding › Conference Article in proceeding › Academic › peer-review

TY - GEN

T1 - Real-Time Multimodal Feedback with the CPR Tutor

AU - Di Mitri, D.

AU - Schneider Barnes, J.

AU - Trebing, Kevin

AU - Sopka, Sasa

AU - Specht, M.M.

AU - Drachsler, H.J.

N1 - Conference code: 21

PY - 2020/6/30

Y1 - 2020/6/30

N2 - We developed the CPR Tutor, a real-time multimodal feedback system for cardiopulmonary resuscitation (CPR) training. The CPR Tutor detects mistakes using recurrent neural networks for real-time time-series classification. From a multimodal data stream consisting of kinematic and electromyographic data, the CPR Tutor system automatically detects the chest compressions, which are then classified and assessed according to five performance indicators. Based on this assessment, the CPR Tutor provides audio feedback to correct the most critical mistakes and improve the CPR performance. To test the validity of the CPR Tutor, we first collected the data corpus from 10 experts used for model training. Hence, to test the impact of the feedback functionality, we ran a user study involving 10 participants. The CPR Tutor pushes forward the current state of the art of real-time multimodal tutors by providing: 1) an architecture design, 2) a methodological approach to design multimodal feedback and 3) a field study on real-time feedback for CPR training.

AB - We developed the CPR Tutor, a real-time multimodal feedback system for cardiopulmonary resuscitation (CPR) training. The CPR Tutor detects mistakes using recurrent neural networks for real-time time-series classification. From a multimodal data stream consisting of kinematic and electromyographic data, the CPR Tutor system automatically detects the chest compressions, which are then classified and assessed according to five performance indicators. Based on this assessment, the CPR Tutor provides audio feedback to correct the most critical mistakes and improve the CPR performance. To test the validity of the CPR Tutor, we first collected the data corpus from 10 experts used for model training. Hence, to test the impact of the feedback functionality, we ran a user study involving 10 participants. The CPR Tutor pushes forward the current state of the art of real-time multimodal tutors by providing: 1) an architecture design, 2) a methodological approach to design multimodal feedback and 3) a field study on real-time feedback for CPR training.

KW - AI in education

KW - artificial intelligence

KW - cpr tutor

KW - learning analytics

KW - multimodal data

KW - real-time feedback

KW - training

U2 - 10.1007/978-3-030-52237-7_12

DO - 10.1007/978-3-030-52237-7_12

M3 - Conference Article in proceeding

SN - 9783030522360

VL - 1

T3 - Lecture Notes in Computer Science (LNCS) series

SP - 141

EP - 152

BT - Artificial Intelligence in Education

A2 - Bittencourt, Ig Ibert

A2 - Cukurova, Mutlu

A2 - Muldner, Kasia

A2 - Luckin, Rose

A2 - Millán, Eva

PB - Springer International Publishing

CY - Cham

T2 - 21st International Conference on Artificial Intelligence in Education

Y2 - 6 July 2020 through 10 July 2020

ER -

Di Mitri D, Schneider Barnes J, Trebing K, Sopka S, Specht MM, Drachsler HJ. Real-Time Multimodal Feedback with the CPR Tutor. In Bittencourt II, Cukurova M, Muldner K, Luckin R, Millán E, editors, Artificial Intelligence in Education: 21st International Conference, AIED 2020, Ifrane, Morocco, July 6–10, 2020, Proceedings, Part I. Vol. 1. Cham: Springer International Publishing. 2020. p. 141-152. (Lecture Notes in Computer Science (LNCS) series, Vol. 12163). (Lecture Notes in Artificial Intelligence (subseries), Vol. 12163). doi: 10.1007/978-3-030-52237-7_12

Real-Time Multimodal Feedback with the CPR Tutor

Abstract

Publication series

Conference

Keywords

Access to Document

Fingerprint

Cite this