Designing computer-based tests: design guidelines from multimedia learning studied with eye tracking

K.J.H. Dirkx, I.T. Skuballa, Claudia Manastriean, H.M. Jarodzka*

*Corresponding author for this work

Research output: Contribution to journalArticleAcademicpeer-review

Abstract

The use of computer-based tests (CBTs), for both formative and summative purposes, has greatly increased over the past years. One major advantage of CBTs is the easy integration of multimedia. It is unclear, though, how to design such CBT environments with multimedia. The purpose of the current study was to examine whether guidelines for designing multimedia instruction based on the Cognitive Load Theory (CLT) and Cognitive Theory of Multimedia Learning (CTML) will yield similar effects in CBT. In a within-subject design, thirty-three vocational students completed a computer-based arithmetic exam, in which half of the items were presented in an original design format, and the other half was redesigned based on the CTML principles for instructional design (i.e., adapted format). Results showed that applying CTML principles to a CBT decreased the difficulty of the test items, i.e., students scored significantly higher on the adapted test items. Moreover, eye-tracking data showed that the adapted items required less visual search and increased attention for the question and answer. Finally, cognitive load, measured as silent pauses during a secondary think-aloud task, decreased. Mean fixation duration (a different indicator of cognitive load), however, did not significantly differ between adapted and original items. These results indicate that applying multimedia principles to CBTs can be beneficial. It seems to prevent cognitive overload and helps students to focus on important parts of the test items (e.g., the question), leading to better test results.
Original languageEnglish
Pages (from-to)589–605
Number of pages17
JournalInstructional Science
Volume49
Issue number5
DOIs
Publication statusPublished - Oct 2021

Keywords

  • testing
  • computer-based testing
  • eye-tracking
  • mental effort
  • multimedia
  • instructional design
  • MOVEMENTS
  • COGNITIVE-LOAD
  • Mental effort
  • Multimedia learning
  • INFORMATION
  • STUDENTS
  • Design
  • ARCHITECTURE
  • Eye tracking
  • Computer-based testing

Fingerprint

Dive into the research topics of 'Designing computer-based tests: design guidelines from multimedia learning studied with eye tracking'. Together they form a unique fingerprint.

Cite this