Reinforcing Stealth Assessment in Serious Games

Konstantinos Georgiadis*, Giel van Lankveld, Kiavash Bahreini, Wim Westera

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference article in proceedingAcademicpeer-review

Abstract

Stealth assessment is a principled assessment methodology proposed for serious games that uses statistical models and machine learning technology to infer players’ mastery levels from logged gameplay data. Although stealth assessment has been proven to be valid and reliable, its application is complex, laborious, and time-consuming. A generic stealth assessment tool (GSAT), proven for its robustness with simulation data, has been proposed to resolve these issues. In this study, GSAT’s robustness is further investigated by using real-world data collected from a serious game on personality traits and validated with an associated personality questionnaire (NEO PI-R). To achieve this, (a) a stepwise regression approach was followed for generating statistical models from logged data for the big five personality traits (OCEAN model), (b) the statistical models are then used with GSAT to produce inferences regarding learners’ mastery level on these personality traits, and (c) the validity of GSAT’s outcomes are examined through a correlation analysis using the results of the NEO PI-R questionnaire. Despite the small dataset GSAT was capable of making inferences on players’ personality traits. This study has demonstrated the practicable feasibility of the SA methodology with GSAT and provides a showcase for its wider application in serious games.
Original languageEnglish
Title of host publicationGames and Learning Alliance
Subtitle of host publication8th International Conference, GALA 2019, Athens, Greece, November 27–29, 2019, Proceedings
EditorsAntonios Liapis, Georgios N. Yannakakis, Manuel Gentile, Manuel Ninaus
Place of PublicationCham
PublisherSpringer
Chapter49
Pages512-521
Number of pages10
ISBN (Electronic)9783030343507
ISBN (Print)9783030343491
DOIs
Publication statusPublished - Nov 2019
Event8th International Conference on Games and Learning Alliance - Athens, Greece
Duration: 27 Nov 201929 Nov 2019

Publication series

SeriesLecture Notes in Computer Science (LNCS)
Volume11899
ISSN0302-9743

Conference

Conference8th International Conference on Games and Learning Alliance
Abbreviated titleGALA 2019
CountryGreece
CityAthens
Period27/11/1929/11/19

Fingerprint

Learning systems
Serious games
Statistical Models

Cite this

Georgiadis, K., van Lankveld, G., Bahreini, K., & Westera, W. (2019). Reinforcing Stealth Assessment in Serious Games. In A. Liapis, G. N. Yannakakis, M. Gentile, & M. Ninaus (Eds.), Games and Learning Alliance: 8th International Conference, GALA 2019, Athens, Greece, November 27–29, 2019, Proceedings (pp. 512-521). Cham: Springer. Lecture Notes in Computer Science (LNCS), Vol.. 11899 https://doi.org/10.1007/978-3-030-34350-7_49
Georgiadis, Konstantinos ; van Lankveld, Giel ; Bahreini, Kiavash ; Westera, Wim. / Reinforcing Stealth Assessment in Serious Games. Games and Learning Alliance: 8th International Conference, GALA 2019, Athens, Greece, November 27–29, 2019, Proceedings. editor / Antonios Liapis ; Georgios N. Yannakakis ; Manuel Gentile ; Manuel Ninaus. Cham : Springer, 2019. pp. 512-521 (Lecture Notes in Computer Science (LNCS), Vol. 11899).
@inproceedings{196ab8925e684cfa82ced02028fdb144,
title = "Reinforcing Stealth Assessment in Serious Games",
abstract = "Stealth assessment is a principled assessment methodology proposed for serious games that uses statistical models and machine learning technology to infer players’ mastery levels from logged gameplay data. Although stealth assessment has been proven to be valid and reliable, its application is complex, laborious, and time-consuming. A generic stealth assessment tool (GSAT), proven for its robustness with simulation data, has been proposed to resolve these issues. In this study, GSAT’s robustness is further investigated by using real-world data collected from a serious game on personality traits and validated with an associated personality questionnaire (NEO PI-R). To achieve this, (a) a stepwise regression approach was followed for generating statistical models from logged data for the big five personality traits (OCEAN model), (b) the statistical models are then used with GSAT to produce inferences regarding learners’ mastery level on these personality traits, and (c) the validity of GSAT’s outcomes are examined through a correlation analysis using the results of the NEO PI-R questionnaire. Despite the small dataset GSAT was capable of making inferences on players’ personality traits. This study has demonstrated the practicable feasibility of the SA methodology with GSAT and provides a showcase for its wider application in serious games.",
author = "Konstantinos Georgiadis and {van Lankveld}, Giel and Kiavash Bahreini and Wim Westera",
year = "2019",
month = "11",
doi = "10.1007/978-3-030-34350-7_49",
language = "English",
isbn = "9783030343491",
series = "Lecture Notes in Computer Science (LNCS)",
publisher = "Springer",
pages = "512--521",
editor = "Antonios Liapis and Yannakakis, {Georgios N.} and Manuel Gentile and Manuel Ninaus",
booktitle = "Games and Learning Alliance",

}

Georgiadis, K, van Lankveld, G, Bahreini, K & Westera, W 2019, Reinforcing Stealth Assessment in Serious Games. in A Liapis, GN Yannakakis, M Gentile & M Ninaus (eds), Games and Learning Alliance: 8th International Conference, GALA 2019, Athens, Greece, November 27–29, 2019, Proceedings. Springer, Cham, Lecture Notes in Computer Science (LNCS), vol. 11899, pp. 512-521, 8th International Conference on Games and Learning Alliance, Athens, Greece, 27/11/19. https://doi.org/10.1007/978-3-030-34350-7_49

Reinforcing Stealth Assessment in Serious Games. / Georgiadis, Konstantinos; van Lankveld, Giel; Bahreini, Kiavash; Westera, Wim.

Games and Learning Alliance: 8th International Conference, GALA 2019, Athens, Greece, November 27–29, 2019, Proceedings. ed. / Antonios Liapis; Georgios N. Yannakakis; Manuel Gentile; Manuel Ninaus. Cham : Springer, 2019. p. 512-521 (Lecture Notes in Computer Science (LNCS), Vol. 11899).

Research output: Chapter in Book/Report/Conference proceedingConference article in proceedingAcademicpeer-review

TY - GEN

T1 - Reinforcing Stealth Assessment in Serious Games

AU - Georgiadis, Konstantinos

AU - van Lankveld, Giel

AU - Bahreini, Kiavash

AU - Westera, Wim

PY - 2019/11

Y1 - 2019/11

N2 - Stealth assessment is a principled assessment methodology proposed for serious games that uses statistical models and machine learning technology to infer players’ mastery levels from logged gameplay data. Although stealth assessment has been proven to be valid and reliable, its application is complex, laborious, and time-consuming. A generic stealth assessment tool (GSAT), proven for its robustness with simulation data, has been proposed to resolve these issues. In this study, GSAT’s robustness is further investigated by using real-world data collected from a serious game on personality traits and validated with an associated personality questionnaire (NEO PI-R). To achieve this, (a) a stepwise regression approach was followed for generating statistical models from logged data for the big five personality traits (OCEAN model), (b) the statistical models are then used with GSAT to produce inferences regarding learners’ mastery level on these personality traits, and (c) the validity of GSAT’s outcomes are examined through a correlation analysis using the results of the NEO PI-R questionnaire. Despite the small dataset GSAT was capable of making inferences on players’ personality traits. This study has demonstrated the practicable feasibility of the SA methodology with GSAT and provides a showcase for its wider application in serious games.

AB - Stealth assessment is a principled assessment methodology proposed for serious games that uses statistical models and machine learning technology to infer players’ mastery levels from logged gameplay data. Although stealth assessment has been proven to be valid and reliable, its application is complex, laborious, and time-consuming. A generic stealth assessment tool (GSAT), proven for its robustness with simulation data, has been proposed to resolve these issues. In this study, GSAT’s robustness is further investigated by using real-world data collected from a serious game on personality traits and validated with an associated personality questionnaire (NEO PI-R). To achieve this, (a) a stepwise regression approach was followed for generating statistical models from logged data for the big five personality traits (OCEAN model), (b) the statistical models are then used with GSAT to produce inferences regarding learners’ mastery level on these personality traits, and (c) the validity of GSAT’s outcomes are examined through a correlation analysis using the results of the NEO PI-R questionnaire. Despite the small dataset GSAT was capable of making inferences on players’ personality traits. This study has demonstrated the practicable feasibility of the SA methodology with GSAT and provides a showcase for its wider application in serious games.

U2 - 10.1007/978-3-030-34350-7_49

DO - 10.1007/978-3-030-34350-7_49

M3 - Conference article in proceeding

SN - 9783030343491

T3 - Lecture Notes in Computer Science (LNCS)

SP - 512

EP - 521

BT - Games and Learning Alliance

A2 - Liapis, Antonios

A2 - Yannakakis, Georgios N.

A2 - Gentile, Manuel

A2 - Ninaus, Manuel

PB - Springer

CY - Cham

ER -

Georgiadis K, van Lankveld G, Bahreini K, Westera W. Reinforcing Stealth Assessment in Serious Games. In Liapis A, Yannakakis GN, Gentile M, Ninaus M, editors, Games and Learning Alliance: 8th International Conference, GALA 2019, Athens, Greece, November 27–29, 2019, Proceedings. Cham: Springer. 2019. p. 512-521. (Lecture Notes in Computer Science (LNCS), Vol. 11899). https://doi.org/10.1007/978-3-030-34350-7_49