Multimodal Challenge: Analytics Beyond User-computer Interaction Data

Daniele Di Mitri, Jan Schneider, Marcus Specht, Hendrik Drachsler

    Research output: Chapter in Book/Report/Conference proceedingConference article in proceedingAcademicpeer-review

    6 Downloads (Pure)


    This contribution describes one the challenges explored in the Fourth LAK Hackathon. This challenge aims at shifting the focus from learning situations which can be easily traced through user-computer interactions data and concentrate more on user-world interactions events, typical of co-located and practice-based learning experiences. This mission, pursued by the multimodal learning analytics (MMLA) community, seeks to bridge the gap between digital and physical learning spaces. The “multimodal” approach consists in combining learners’ motoric actions with physiological responses and data about the learning contexts. These data can be collected through multiple wearable sensors and Internet of Things (IoT) devices. This Hackathon table will confront with three main challenges arising from the analysis and valorisation of multimodal datasets: 1) the data collection and storing, 2) the data annotation, 3) the data processing and exploitation. Some research questions which will be considered in this Hackathon challenge are the following: how to process the raw sensor data streams and extract relevant features? which data mining and machine learning techniques can be applied? how can we compare two action recordings? How to combine sensor data with Experience API (xAPI)? what are meaningful visualisations for these data?
    Original languageEnglish
    Title of host publicationCompanion Proceedings of the 8th International Conference on Learning Analytics and Knowledge (LAK'18)
    Subtitle of host publicationTowards User-Centred Learning Analytics
    EditorsAbelardo Pardo, Kathryn Bartimote, Grace Lynch, Simon Buckingham Shum, Rebecca Ferguson, Agathe Merceron, Xavier Ochoa
    Place of PublicationSydney, Australia
    Number of pages4
    Publication statusPublished - Mar 2018
    EventThe 8th International Learning Analytics & Knowledge Conference - SMC Conference & Function Centre, Sydney, Australia
    Duration: 5 Mar 20189 Mar 2018


    ConferenceThe 8th International Learning Analytics & Knowledge Conference
    Abbreviated titleLAK18
    Internet address


    • multimodal learning analytics
    • wearables
    • CrossMMLA
    • sensor-based learning


    Dive into the research topics of 'Multimodal Challenge: Analytics Beyond User-computer Interaction Data'. Together they form a unique fingerprint.

    Cite this