Bring It on! Challenges Encountered While Building a Comprehensive Tutoring System Using ReaderBench

Marilena Panaite, Mihai Dascalu, Amy M. Johnson, Renu Balyan, Jianmin Dai, Danielle S. McNamara, Stefan Trausan-Matu

    Research output: Chapter in Book/Report/Conference proceedingConference article in proceedingAcademicpeer-review

    24 Downloads (Pure)

    Abstract

    Intelligent Tutoring Systems (ITSs) are aimed at promoting acquisition of knowledge and skills by providing relevant and appropriate feedback during students’ practice activities. ITSs for literacy instruction commonly assess typed responses using Natural Language Processing (NLP) algorithms. One step in this direction often requires building a scoring mechanism that matches human judgments. This paper describes the challenges encountered while implementing an automated evaluation workflow and adopting solutions for increasing performance of the tutoring system. The algorithm described here comprises multiple stages, including initial pre-processing, a rule-based system for pre-classifying self-explanations, followed by classification using a Support Virtual Machine (SVM) learning algorithm. The SVM model hyper-parameters were optimized using grid search approach with 4,109 different self explanations scored 0 to 3 (i.e., poor to great). The accuracy achieved for the model was 59% (adjacent accuracy = 97%; Kappa = .43).
    Original languageEnglish
    Title of host publicationArtificial Intelligence in Education
    Subtitle of host publication19th International Conference, AIED 2018, London, UK, June 27-30, 2018, Proceedings, Part I
    EditorsC. P. Rosé, R. Martínez-Maldonado, H. U. Hoppe, R. Luckin, M. Mavrikis, K. Porayska-Pomsta, B. McLaren, B. d. Boulay
    Place of PublicationCham
    PublisherSpringer International Publishing AG
    Pages409-419
    Number of pages11
    ISBN (Electronic)9783319938431
    ISBN (Print)9783319938424
    DOIs
    Publication statusPublished - 2018
    EventInternational Conference, AIED 2018 - London, United Kingdom
    Duration: 27 Jun 201830 Jun 2018
    https://aied2018.utscic.edu.au/

    Publication series

    SeriesLecture Notes in Computer Science
    Volume10947
    SeriesLecture Notes in Artificial Intelligence (subseries)

    Conference

    ConferenceInternational Conference, AIED 2018
    Abbreviated titleAIED 2018
    CountryUnited Kingdom
    CityLondon
    Period27/06/1830/06/18
    Internet address

      Fingerprint

    Keywords

    • Natural language processing
    • Intelligent tutoring systems
    • Self-explanations
    • Support vector machines
    • ReaderBench

    Cite this

    Panaite, M., Dascalu, M., Johnson, A. M., Balyan, R., Dai, J., McNamara, D. S., & Trausan-Matu, S. (2018). Bring It on! Challenges Encountered While Building a Comprehensive Tutoring System Using ReaderBench. In C. P. Rosé, R. Martínez-Maldonado, H. U. Hoppe, R. Luckin, M. Mavrikis, K. Porayska-Pomsta, B. McLaren, & B. D. Boulay (Eds.), Artificial Intelligence in Education: 19th International Conference, AIED 2018, London, UK, June 27-30, 2018, Proceedings, Part I (pp. 409-419). Springer International Publishing AG. Lecture Notes in Computer Science, Vol.. 10947, Lecture Notes in Artificial Intelligence (subseries) https://doi.org/10.1007/978-3-319-93843-1_30