The Dilemmas of Formulating Theory-Informed Design Guidelines for a Video Enhanced Rubric for the Formative Assessment of Complex Skills

Kevin Ackermans*, Ellen Rusman, Saskia Brand-Gruwel, Marcus Specht

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference article in proceedingAcademicpeer-review

3 Downloads (Pure)

Abstract

Learners aiming to master a complex skill may benefit from the combina-tion of abstract information found in a text-based analytical rubric and con-crete information provided by a video modeling example. In this paper, we address the design dilemmas of combining video modeling examples and rubrics into a Video Enhanced Rubric. We propose a model to address these design dilemma's and develop our first prototype based on this model. We review the first prototype through a two-stage international expert valida-tion session. In the first stage, 20 experts are asked to design a user inter-face for the Video Enhanced Rubric. In the second stage, 20 experts are asked to perform an expert appraisal of our first prototype. The preliminary results of the expert validation session are subsequently analyzed using Sauli, Cattaneo and van der Meij’s Framework for Developing Instructional Hypervideo to detect common design suggestions. Following the results of the expert validation, we developed a second prototype of the Video En-hanced Rubric. With the design guidelines of a Video Enhanced Rubric, we aim to improve the formative assessment and mastery of complex skills by fostering learner’s mental model development and the quality (consistency, concreteness) of both given as well as received feedback. On a more general note, we expect the design dilemmas addressed in this paper to inform re-searchers who aim to apply theoretical multimedia design guidelines to formative assessment practices with rubrics.
Original languageEnglish
Title of host publicationTechnology Enhanced Assessment
Subtitle of host publication20th International Conference, TEA 2017, Barcelona, Spain, October 5–6, 2017, Revised Selected Papers
EditorsE. Ras, E. G. R. Ana
Place of PublicationCham, Switserland
PublisherSpringer
Pages123-136
Volume654
DOIs
Publication statusPublished - 5 Dec 2017
EventInternational Technology Enhanced Assessment Conference - Barcelona, Spain
Duration: 5 Oct 20176 Oct 2017

Publication series

NameCommunications in Computer and Information Science
PublisherSpringer
Volume829
ISSN (Print)1865-0929
ISSN (Electronic)1865-0937

Conference

ConferenceInternational Technology Enhanced Assessment Conference
CountrySpain
CityBarcelona
Period5/10/176/10/17

Fingerprint

Feedback

Keywords

  • Video Enhanced Rubrics
  • Rubrics
  • Multimedia

Cite this

Ackermans, K., Rusman, E., Brand-Gruwel, S., & Specht, M. (2017). The Dilemmas of Formulating Theory-Informed Design Guidelines for a Video Enhanced Rubric for the Formative Assessment of Complex Skills. In E. Ras, & E. G. R. Ana (Eds.), Technology Enhanced Assessment: 20th International Conference, TEA 2017, Barcelona, Spain, October 5–6, 2017, Revised Selected Papers (Vol. 654, pp. 123-136). (Communications in Computer and Information Science; Vol. 829). Cham, Switserland: Springer. https://doi.org/10.1007/978-3-319-97807-9_10
Ackermans, Kevin ; Rusman, Ellen ; Brand-Gruwel, Saskia ; Specht, Marcus. / The Dilemmas of Formulating Theory-Informed Design Guidelines for a Video Enhanced Rubric for the Formative Assessment of Complex Skills. Technology Enhanced Assessment: 20th International Conference, TEA 2017, Barcelona, Spain, October 5–6, 2017, Revised Selected Papers. editor / E. Ras ; E. G. R. Ana. Vol. 654 Cham, Switserland : Springer, 2017. pp. 123-136 (Communications in Computer and Information Science).
@inproceedings{cdc779f636cd41d489699a1c3588d164,
title = "The Dilemmas of Formulating Theory-Informed Design Guidelines for a Video Enhanced Rubric for the Formative Assessment of Complex Skills",
abstract = "Learners aiming to master a complex skill may benefit from the combina-tion of abstract information found in a text-based analytical rubric and con-crete information provided by a video modeling example. In this paper, we address the design dilemmas of combining video modeling examples and rubrics into a Video Enhanced Rubric. We propose a model to address these design dilemma's and develop our first prototype based on this model. We review the first prototype through a two-stage international expert valida-tion session. In the first stage, 20 experts are asked to design a user inter-face for the Video Enhanced Rubric. In the second stage, 20 experts are asked to perform an expert appraisal of our first prototype. The preliminary results of the expert validation session are subsequently analyzed using Sauli, Cattaneo and van der Meij’s Framework for Developing Instructional Hypervideo to detect common design suggestions. Following the results of the expert validation, we developed a second prototype of the Video En-hanced Rubric. With the design guidelines of a Video Enhanced Rubric, we aim to improve the formative assessment and mastery of complex skills by fostering learner’s mental model development and the quality (consistency, concreteness) of both given as well as received feedback. On a more general note, we expect the design dilemmas addressed in this paper to inform re-searchers who aim to apply theoretical multimedia design guidelines to formative assessment practices with rubrics.",
keywords = "Video Enhanced Rubrics, Rubrics, Multimedia",
author = "Kevin Ackermans and Ellen Rusman and Saskia Brand-Gruwel and Marcus Specht",
year = "2017",
month = "12",
day = "5",
doi = "10.1007/978-3-319-97807-9_10",
language = "English",
volume = "654",
series = "Communications in Computer and Information Science",
publisher = "Springer",
pages = "123--136",
editor = "E. Ras and Ana, {E. G. R.}",
booktitle = "Technology Enhanced Assessment",

}

Ackermans, K, Rusman, E, Brand-Gruwel, S & Specht, M 2017, The Dilemmas of Formulating Theory-Informed Design Guidelines for a Video Enhanced Rubric for the Formative Assessment of Complex Skills. in E Ras & EGR Ana (eds), Technology Enhanced Assessment: 20th International Conference, TEA 2017, Barcelona, Spain, October 5–6, 2017, Revised Selected Papers. vol. 654, Communications in Computer and Information Science, vol. 829, Springer, Cham, Switserland, pp. 123-136, International Technology Enhanced Assessment Conference, Barcelona, Spain, 5/10/17. https://doi.org/10.1007/978-3-319-97807-9_10

The Dilemmas of Formulating Theory-Informed Design Guidelines for a Video Enhanced Rubric for the Formative Assessment of Complex Skills. / Ackermans, Kevin; Rusman, Ellen; Brand-Gruwel, Saskia; Specht, Marcus.

Technology Enhanced Assessment: 20th International Conference, TEA 2017, Barcelona, Spain, October 5–6, 2017, Revised Selected Papers. ed. / E. Ras; E. G. R. Ana. Vol. 654 Cham, Switserland : Springer, 2017. p. 123-136 (Communications in Computer and Information Science; Vol. 829).

Research output: Chapter in Book/Report/Conference proceedingConference article in proceedingAcademicpeer-review

TY - GEN

T1 - The Dilemmas of Formulating Theory-Informed Design Guidelines for a Video Enhanced Rubric for the Formative Assessment of Complex Skills

AU - Ackermans, Kevin

AU - Rusman, Ellen

AU - Brand-Gruwel, Saskia

AU - Specht, Marcus

PY - 2017/12/5

Y1 - 2017/12/5

N2 - Learners aiming to master a complex skill may benefit from the combina-tion of abstract information found in a text-based analytical rubric and con-crete information provided by a video modeling example. In this paper, we address the design dilemmas of combining video modeling examples and rubrics into a Video Enhanced Rubric. We propose a model to address these design dilemma's and develop our first prototype based on this model. We review the first prototype through a two-stage international expert valida-tion session. In the first stage, 20 experts are asked to design a user inter-face for the Video Enhanced Rubric. In the second stage, 20 experts are asked to perform an expert appraisal of our first prototype. The preliminary results of the expert validation session are subsequently analyzed using Sauli, Cattaneo and van der Meij’s Framework for Developing Instructional Hypervideo to detect common design suggestions. Following the results of the expert validation, we developed a second prototype of the Video En-hanced Rubric. With the design guidelines of a Video Enhanced Rubric, we aim to improve the formative assessment and mastery of complex skills by fostering learner’s mental model development and the quality (consistency, concreteness) of both given as well as received feedback. On a more general note, we expect the design dilemmas addressed in this paper to inform re-searchers who aim to apply theoretical multimedia design guidelines to formative assessment practices with rubrics.

AB - Learners aiming to master a complex skill may benefit from the combina-tion of abstract information found in a text-based analytical rubric and con-crete information provided by a video modeling example. In this paper, we address the design dilemmas of combining video modeling examples and rubrics into a Video Enhanced Rubric. We propose a model to address these design dilemma's and develop our first prototype based on this model. We review the first prototype through a two-stage international expert valida-tion session. In the first stage, 20 experts are asked to design a user inter-face for the Video Enhanced Rubric. In the second stage, 20 experts are asked to perform an expert appraisal of our first prototype. The preliminary results of the expert validation session are subsequently analyzed using Sauli, Cattaneo and van der Meij’s Framework for Developing Instructional Hypervideo to detect common design suggestions. Following the results of the expert validation, we developed a second prototype of the Video En-hanced Rubric. With the design guidelines of a Video Enhanced Rubric, we aim to improve the formative assessment and mastery of complex skills by fostering learner’s mental model development and the quality (consistency, concreteness) of both given as well as received feedback. On a more general note, we expect the design dilemmas addressed in this paper to inform re-searchers who aim to apply theoretical multimedia design guidelines to formative assessment practices with rubrics.

KW - Video Enhanced Rubrics

KW - Rubrics

KW - Multimedia

U2 - 10.1007/978-3-319-97807-9_10

DO - 10.1007/978-3-319-97807-9_10

M3 - Conference article in proceeding

VL - 654

T3 - Communications in Computer and Information Science

SP - 123

EP - 136

BT - Technology Enhanced Assessment

A2 - Ras, E.

A2 - Ana, E. G. R.

PB - Springer

CY - Cham, Switserland

ER -

Ackermans K, Rusman E, Brand-Gruwel S, Specht M. The Dilemmas of Formulating Theory-Informed Design Guidelines for a Video Enhanced Rubric for the Formative Assessment of Complex Skills. In Ras E, Ana EGR, editors, Technology Enhanced Assessment: 20th International Conference, TEA 2017, Barcelona, Spain, October 5–6, 2017, Revised Selected Papers. Vol. 654. Cham, Switserland: Springer. 2017. p. 123-136. (Communications in Computer and Information Science). https://doi.org/10.1007/978-3-319-97807-9_10