A methodology for assessing the effectiveness of serious games and for inferring player learning outcomes.

Ángel Serrano-Laguna, Borja Manero, Manuel Freire, Baltasar Fernández Manjón

    Research output: Contribution to journalArticleAcademicpeer-review

    10 Downloads (Pure)

    Abstract

    Although serious games are proven to serve as educational tools in many educational domains, there is a lack of reliable, automated and repeatable methodologies that measure their effectiveness: what do players know after playing serious games? Do they learn from them? Previous research shows that the vast majority of serious games are assessed by using questionnaires, which is in stark contrast to current trends in the video game industry. Commercial videogame developers have been learning from their players through Game Analytics for years via non-disruptive game tracking. In this paper, we propose a methodology for assessing serious game effectiveness based on non-disruptive in-game tracking. The methodology involves a design pattern that structures the delivery of educational goals through a game. This structure also allows one to infer learning outcomes for each individual player, which, when aggregated, determine the effectiveness of a serious game. We tested the methodology by having 320 students play a serious game. The proposed methodology allowed us to infer players' learning outcomes, to assess the game effectiveness levels and to identify issues in the game design.

    Original languageEnglish
    Pages (from-to)2849-2871
    Number of pages23
    JournalMultimedia Tools and Applications
    Volume77
    Issue number2
    DOIs
    Publication statusPublished - Jan 2018

    Fingerprint

    Serious games
    Students
    Industry

    Keywords

    • Serious games
    • Learning analytics
    • Game design
    • Learning outcomes analysis
    • Educational games
    • DESIGN
    • ANALYTICS
    • FRAMEWORK
    • STUDENTS
    • MODEL

    Cite this

    Serrano-Laguna, Ángel ; Manero, Borja ; Freire, Manuel ; Fernández Manjón, Baltasar. / A methodology for assessing the effectiveness of serious games and for inferring player learning outcomes. In: Multimedia Tools and Applications. 2018 ; Vol. 77, No. 2. pp. 2849-2871.
    @article{d1de033e05af4da488a03225e6f3e7d4,
    title = "A methodology for assessing the effectiveness of serious games and for inferring player learning outcomes.",
    abstract = "Although serious games are proven to serve as educational tools in many educational domains, there is a lack of reliable, automated and repeatable methodologies that measure their effectiveness: what do players know after playing serious games? Do they learn from them? Previous research shows that the vast majority of serious games are assessed by using questionnaires, which is in stark contrast to current trends in the video game industry. Commercial videogame developers have been learning from their players through Game Analytics for years via non-disruptive game tracking. In this paper, we propose a methodology for assessing serious game effectiveness based on non-disruptive in-game tracking. The methodology involves a design pattern that structures the delivery of educational goals through a game. This structure also allows one to infer learning outcomes for each individual player, which, when aggregated, determine the effectiveness of a serious game. We tested the methodology by having 320 students play a serious game. The proposed methodology allowed us to infer players' learning outcomes, to assess the game effectiveness levels and to identify issues in the game design.",
    keywords = "Serious games, Learning analytics, Game design, Learning outcomes analysis, Educational games, DESIGN, ANALYTICS, FRAMEWORK, STUDENTS, MODEL",
    author = "{\'A}ngel Serrano-Laguna and Borja Manero and Manuel Freire and {Fern{\'a}ndez Manj{\'o}n}, Baltasar",
    year = "2018",
    month = "1",
    doi = "10.1007/s11042-017-4467-6",
    language = "English",
    volume = "77",
    pages = "2849--2871",
    journal = "Multimedia Tools and Applications",
    issn = "1380-7501",
    publisher = "Springer Netherlands",
    number = "2",

    }

    A methodology for assessing the effectiveness of serious games and for inferring player learning outcomes. / Serrano-Laguna, Ángel; Manero, Borja; Freire, Manuel; Fernández Manjón, Baltasar.

    In: Multimedia Tools and Applications, Vol. 77, No. 2, 01.2018, p. 2849-2871.

    Research output: Contribution to journalArticleAcademicpeer-review

    TY - JOUR

    T1 - A methodology for assessing the effectiveness of serious games and for inferring player learning outcomes.

    AU - Serrano-Laguna, Ángel

    AU - Manero, Borja

    AU - Freire, Manuel

    AU - Fernández Manjón, Baltasar

    PY - 2018/1

    Y1 - 2018/1

    N2 - Although serious games are proven to serve as educational tools in many educational domains, there is a lack of reliable, automated and repeatable methodologies that measure their effectiveness: what do players know after playing serious games? Do they learn from them? Previous research shows that the vast majority of serious games are assessed by using questionnaires, which is in stark contrast to current trends in the video game industry. Commercial videogame developers have been learning from their players through Game Analytics for years via non-disruptive game tracking. In this paper, we propose a methodology for assessing serious game effectiveness based on non-disruptive in-game tracking. The methodology involves a design pattern that structures the delivery of educational goals through a game. This structure also allows one to infer learning outcomes for each individual player, which, when aggregated, determine the effectiveness of a serious game. We tested the methodology by having 320 students play a serious game. The proposed methodology allowed us to infer players' learning outcomes, to assess the game effectiveness levels and to identify issues in the game design.

    AB - Although serious games are proven to serve as educational tools in many educational domains, there is a lack of reliable, automated and repeatable methodologies that measure their effectiveness: what do players know after playing serious games? Do they learn from them? Previous research shows that the vast majority of serious games are assessed by using questionnaires, which is in stark contrast to current trends in the video game industry. Commercial videogame developers have been learning from their players through Game Analytics for years via non-disruptive game tracking. In this paper, we propose a methodology for assessing serious game effectiveness based on non-disruptive in-game tracking. The methodology involves a design pattern that structures the delivery of educational goals through a game. This structure also allows one to infer learning outcomes for each individual player, which, when aggregated, determine the effectiveness of a serious game. We tested the methodology by having 320 students play a serious game. The proposed methodology allowed us to infer players' learning outcomes, to assess the game effectiveness levels and to identify issues in the game design.

    KW - Serious games

    KW - Learning analytics

    KW - Game design

    KW - Learning outcomes analysis

    KW - Educational games

    KW - DESIGN

    KW - ANALYTICS

    KW - FRAMEWORK

    KW - STUDENTS

    KW - MODEL

    U2 - 10.1007/s11042-017-4467-6

    DO - 10.1007/s11042-017-4467-6

    M3 - Article

    VL - 77

    SP - 2849

    EP - 2871

    JO - Multimedia Tools and Applications

    JF - Multimedia Tools and Applications

    SN - 1380-7501

    IS - 2

    ER -