Making Use of Students’ Usage and Perception Information to Improve Online Learning Activities: A Dashboard Design Founded on Teachers’ Needs.

Abstract

Making Use of Students’ Usage and Perception Information to Improve Online Learning Activities:
A Dashboard Design Founded on Teachers’ Needs
L.M. ten Den

Nowadays online learning activities in higher education are common. Teachers at Hogeschool Windesheim receive, in contrast with face-to-face activities, little feedback on students’ use and per-ception of online learning activities, leaving them with insufficient information to improve those activ-ities.
Introduction of log data from Learning Management Systems has expanded the collection of student-data enormously, however there is little evidence that learning analytics are deployed widely in higher educational practice. Besides that, few have actually asked the question what kind of data would be valuable to analyse (Viberg et al, 2018). Ifenthaler (2015) calls on higher education institu-tions to analyse and create new interventions and actions based on LA. Related work has shown teachers’ interest in a wide variety of usage information and students’ willingness to provide feedback (Schmitz et al. 2018). Few studies asked teachers what would help them to enhance their work, even though starting from teachers’ needs is key to create support for the use of LA-tools.
This study aims to identify what information teachers need to be able to improve their online learning activities and to design and evaluate a dashboard that presents data to teachers. In a mixed method design a self-composed survey was distributed to 773 teachers at the institute of higher educa-tion Windesheim (n=77). The survey identifies the learning activities currently used and teachers’ needs for information on students’ (a) use and (b) perception of these learning activities. Using the results of the survey a mock-up prototype was designed using a data-visualisation program called Power Bi. A first and a second prototype was each introduced to 5 teachers who were observed while using it and an evaluation was done with semi-structured questions and the Evaluation Framework for Learning Analytics (Scheffel et al, 2017).
Results from the – not generalisable - survey showed that the four most used learning activi-ties are reading texts, videos, formative assignments, and lectures. On students’ use respondents most-ly would like basic information on how often links to texts and videos are used, assignments are sub-mitted, and lectures attended. Besides that, even more respondents would like information on how students perceive the learning activities. The preferred method for collecting this perception infor-mation is through questions collecting free text answers. In fact, 90% of respondents would use a dashboard with this information.
Results from interviews on the first prototype show that interviewees are enthusiastic about the dashboard and suggested adding effectivity measures such as test results and further clarification on how data was collected. Interviews on the second prototype again bore enthusiastic interviewees and suggestions for two improvements: The first one on using a better technique to show test-results and the second one to add cross-connections per student (still anonymously) to be able to value re-marks and reviews. Finally, evaluation (EFLA) shows a score of 74.1 for the second prototype. An average score of 9.2 on “This dashboard makes me aware” and “This dashboard stimulates me to adapt my learning activities” supports the use of such a dashboard for improving learning activities.

Date of Award	9 Apr 2021
Original language	English
Supervisor	Maren Scheffel (Supervisor)