Bolstering Stealth Assessment in Serious Games

Konstantinos Georgiadis*, Tjitske Faber, Wim Westera

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference article in proceedingAcademicpeer-review

Abstract

Stealth assessment is an unobtrusive assessment methodology in serious games that use digital player traces to make inferences of players’ expertise level over competencies. Although various proofs of stealth assessment’s validity have been reported, its application is a complex, laborious, and time-consuming process. To bolster the applicability of stealth assessment in serious games; a generic stealth assessment tool (GSAT) has been proposed, which uses machine learning techniques to reason over competence constructs, player log data and assess player performance. Current study provides empirical validation of GSAT by applying it to a real-world game, the abcdeSIM game, which was designed to train medical care workers to act effectively medical emergency situations. GSAT demonstrated, while relying on a Gaussian Naive Bayes Network, to be highly robust and reliable, achieving a three-level assessment accuracy of 96%, as compared with a reference score model defined by experts. By this result, this study contributes to the alleviation of stealth assessment’s applicability issues and hence promotes its wider uptake by the serious game community.
Original languageEnglish
Title of host publicationGames and Learning Alliance
Subtitle of host publication8th International Conference, GALA 2019, Athens, Greece, November 27–29, 2019, Proceedings
EditorsAntonios Liapis, Georgios N. Yannakakis, Manuel Gentile, Manuel Ninaus
Place of PublicationCham
PublisherSpringer
Chapter21
Pages211-220
Number of pages10
ISBN (Electronic)9783030343507
ISBN (Print)9783030343491
DOIs
Publication statusPublished - 27 Nov 2019
Event8th International Conference on Games and Learning Alliance - Athens, Greece
Duration: 27 Nov 201929 Nov 2019

Publication series

SeriesLecture Notes in Computer Science (LNCS)
Volume11899
ISSN0302-9743

Conference

Conference8th International Conference on Games and Learning Alliance
Abbreviated titleGALA 2019
CountryGreece
CityAthens
Period27/11/1929/11/19

Fingerprint

Health care
Learning systems
Serious games

Cite this

Georgiadis, K., Faber, T., & Westera, W. (2019). Bolstering Stealth Assessment in Serious Games. In A. Liapis, G. N. Yannakakis, M. Gentile, & M. N. (Eds.), Games and Learning Alliance: 8th International Conference, GALA 2019, Athens, Greece, November 27–29, 2019, Proceedings (pp. 211-220). Cham: Springer. Lecture Notes in Computer Science (LNCS), Vol.. 11899 https://doi.org/10.1007/978-3-030-34350-7_21
Georgiadis, Konstantinos ; Faber, Tjitske ; Westera, Wim. / Bolstering Stealth Assessment in Serious Games. Games and Learning Alliance: 8th International Conference, GALA 2019, Athens, Greece, November 27–29, 2019, Proceedings. editor / Antonios Liapis ; Georgios N. Yannakakis ; Manuel Gentile ; Manuel Ninaus. Cham : Springer, 2019. pp. 211-220 (Lecture Notes in Computer Science (LNCS), Vol. 11899).
@inproceedings{e7db1ce450b944069482a527a5e0e581,
title = "Bolstering Stealth Assessment in Serious Games",
abstract = "Stealth assessment is an unobtrusive assessment methodology in serious games that use digital player traces to make inferences of players’ expertise level over competencies. Although various proofs of stealth assessment’s validity have been reported, its application is a complex, laborious, and time-consuming process. To bolster the applicability of stealth assessment in serious games; a generic stealth assessment tool (GSAT) has been proposed, which uses machine learning techniques to reason over competence constructs, player log data and assess player performance. Current study provides empirical validation of GSAT by applying it to a real-world game, the abcdeSIM game, which was designed to train medical care workers to act effectively medical emergency situations. GSAT demonstrated, while relying on a Gaussian Naive Bayes Network, to be highly robust and reliable, achieving a three-level assessment accuracy of 96{\%}, as compared with a reference score model defined by experts. By this result, this study contributes to the alleviation of stealth assessment’s applicability issues and hence promotes its wider uptake by the serious game community.",
author = "Konstantinos Georgiadis and Tjitske Faber and Wim Westera",
year = "2019",
month = "11",
day = "27",
doi = "10.1007/978-3-030-34350-7_21",
language = "English",
isbn = "9783030343491",
series = "Lecture Notes in Computer Science (LNCS)",
publisher = "Springer",
pages = "211--220",
editor = "Antonios Liapis and Yannakakis, {Georgios N.} and Manuel Gentile and {Manuel Ninaus}",
booktitle = "Games and Learning Alliance",

}

Georgiadis, K, Faber, T & Westera, W 2019, Bolstering Stealth Assessment in Serious Games. in A Liapis, GN Yannakakis, M Gentile & MN (eds), Games and Learning Alliance: 8th International Conference, GALA 2019, Athens, Greece, November 27–29, 2019, Proceedings. Springer, Cham, Lecture Notes in Computer Science (LNCS), vol. 11899, pp. 211-220, 8th International Conference on Games and Learning Alliance, Athens, Greece, 27/11/19. https://doi.org/10.1007/978-3-030-34350-7_21

Bolstering Stealth Assessment in Serious Games. / Georgiadis, Konstantinos ; Faber, Tjitske; Westera, Wim.

Games and Learning Alliance: 8th International Conference, GALA 2019, Athens, Greece, November 27–29, 2019, Proceedings. ed. / Antonios Liapis; Georgios N. Yannakakis; Manuel Gentile; Manuel Ninaus. Cham : Springer, 2019. p. 211-220 (Lecture Notes in Computer Science (LNCS), Vol. 11899).

Research output: Chapter in Book/Report/Conference proceedingConference article in proceedingAcademicpeer-review

TY - GEN

T1 - Bolstering Stealth Assessment in Serious Games

AU - Georgiadis, Konstantinos

AU - Faber, Tjitske

AU - Westera, Wim

PY - 2019/11/27

Y1 - 2019/11/27

N2 - Stealth assessment is an unobtrusive assessment methodology in serious games that use digital player traces to make inferences of players’ expertise level over competencies. Although various proofs of stealth assessment’s validity have been reported, its application is a complex, laborious, and time-consuming process. To bolster the applicability of stealth assessment in serious games; a generic stealth assessment tool (GSAT) has been proposed, which uses machine learning techniques to reason over competence constructs, player log data and assess player performance. Current study provides empirical validation of GSAT by applying it to a real-world game, the abcdeSIM game, which was designed to train medical care workers to act effectively medical emergency situations. GSAT demonstrated, while relying on a Gaussian Naive Bayes Network, to be highly robust and reliable, achieving a three-level assessment accuracy of 96%, as compared with a reference score model defined by experts. By this result, this study contributes to the alleviation of stealth assessment’s applicability issues and hence promotes its wider uptake by the serious game community.

AB - Stealth assessment is an unobtrusive assessment methodology in serious games that use digital player traces to make inferences of players’ expertise level over competencies. Although various proofs of stealth assessment’s validity have been reported, its application is a complex, laborious, and time-consuming process. To bolster the applicability of stealth assessment in serious games; a generic stealth assessment tool (GSAT) has been proposed, which uses machine learning techniques to reason over competence constructs, player log data and assess player performance. Current study provides empirical validation of GSAT by applying it to a real-world game, the abcdeSIM game, which was designed to train medical care workers to act effectively medical emergency situations. GSAT demonstrated, while relying on a Gaussian Naive Bayes Network, to be highly robust and reliable, achieving a three-level assessment accuracy of 96%, as compared with a reference score model defined by experts. By this result, this study contributes to the alleviation of stealth assessment’s applicability issues and hence promotes its wider uptake by the serious game community.

U2 - 10.1007/978-3-030-34350-7_21

DO - 10.1007/978-3-030-34350-7_21

M3 - Conference article in proceeding

SN - 9783030343491

T3 - Lecture Notes in Computer Science (LNCS)

SP - 211

EP - 220

BT - Games and Learning Alliance

A2 - Liapis, Antonios

A2 - Yannakakis, Georgios N.

A2 - Gentile, Manuel

A2 - , Manuel Ninaus

PB - Springer

CY - Cham

ER -

Georgiadis K, Faber T, Westera W. Bolstering Stealth Assessment in Serious Games. In Liapis A, Yannakakis GN, Gentile M, MN, editors, Games and Learning Alliance: 8th International Conference, GALA 2019, Athens, Greece, November 27–29, 2019, Proceedings. Cham: Springer. 2019. p. 211-220. (Lecture Notes in Computer Science (LNCS), Vol. 11899). https://doi.org/10.1007/978-3-030-34350-7_21