The Proof of the Pudding: Examining Validity and Reliability of the Evaluation Framework for Learning Analytics

Maren Scheffel*, Hendrik Drachsler, Christian Toisoul, Stefaan Ternier, Marcus Specht

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference article in proceedingAcademicpeer-review

63 Downloads (Pure)

Abstract

While learning analytics (LA) is maturing from being a trend to being part of the institutional toolbox, the need for more empirical evidences about the effects for LA on the actual stakeholders, i.e. learners and teachers, is increasing. Within this paper we report about a further evaluation iteration of the Evaluation Framework for Learning Analytics (EFLA) that provides an efficient and effective measure to get insights into the application of LA in educational institutes. For this empirical study we have thus developed and implemented several LA widgets into a MOOC platform’s dashboard and evaluated these widgets using the EFLA as well as the framework itself using principal component and reliability analysis. The results show that the EFLA is able to measure differences between widget versions. Furthermore, they indicate that the framework is highly reliable after slightly adapting its dimensions.
Original languageEnglish
Title of host publicationData Driven Approaches in Digital Education
Subtitle of host publicationProceedings of 12th European Conference on Technology Enhanced Learning EC-TEL 2017
EditorsÉ. Lavoué, H. Drachsler, K. Verbert, J. Broisin, M. Pérez-Sanagustín
PublisherSpringer
Pages194-208
ISBN (Electronic)978-3-319-66610-5
ISBN (Print)978-3-319-66609-9
DOIs
Publication statusPublished - 1 Sep 2017
Event12th European Conference on Technology Enhanced Learning: Data Driven Approaches in Digital Education - Tallinn, Estonia
Duration: 12 Sep 201715 Sep 2017
http://ectel2017.httc.de/index.php?id=777

Publication series

SeriesLecture Notes in Computer Science
Volume10474
ISSN0302-9743

Conference

Conference12th European Conference on Technology Enhanced Learning
Abbreviated titleEC-TEL 2017
CountryEstonia
CityTallinn
Period12/09/1715/09/17
Internet address

Keywords

  • learning analytics
  • evaluation
  • validity
  • reliability

Fingerprint Dive into the research topics of 'The Proof of the Pudding: Examining Validity and Reliability of the Evaluation Framework for Learning Analytics'. Together they form a unique fingerprint.

  • Cite this

    Scheffel, M., Drachsler, H., Toisoul, C., Ternier, S., & Specht, M. (2017). The Proof of the Pudding: Examining Validity and Reliability of the Evaluation Framework for Learning Analytics. In É. Lavoué, H. Drachsler, K. Verbert, J. Broisin, & M. Pérez-Sanagustín (Eds.), Data Driven Approaches in Digital Education: Proceedings of 12th European Conference on Technology Enhanced Learning EC-TEL 2017 (pp. 194-208). Springer. Lecture Notes in Computer Science, Vol.. 10474 https://doi.org/10.1007/978-3-319-66610-5_15