Bring It on! Challenges Encountered While Building a Comprehensive Tutoring System Using ReaderBench

Marilena Panaite, Mihai Dascalu, Amy M. Johnson, Renu Balyan, Jianmin Dai, Danielle S. McNamara, Stefan Trausan-Matu

Research output: Chapter in Book/Report/Conference proceedingConference article in proceedingAcademicpeer-review

35 Downloads (Pure)

Abstract

Intelligent Tutoring Systems (ITSs) are aimed at promoting acquisition of knowledge and skills by providing relevant and appropriate feedback during students’ practice activities. ITSs for literacy instruction commonly assess typed responses using Natural Language Processing (NLP) algorithms. One step in this direction often requires building a scoring mechanism that matches human judgments. This paper describes the challenges encountered while implementing an automated evaluation workflow and adopting solutions for increasing performance of the tutoring system. The algorithm described here comprises multiple stages, including initial pre-processing, a rule-based system for pre-classifying self-explanations, followed by classification using a Support Virtual Machine (SVM) learning algorithm. The SVM model hyper-parameters were optimized using grid search approach with 4,109 different self explanations scored 0 to 3 (i.e., poor to great). The accuracy achieved for the model was 59% (adjacent accuracy = 97%; Kappa = .43).
Original languageEnglish
Title of host publicationArtificial Intelligence in Education
Subtitle of host publication19th International Conference, AIED 2018, London, UK, June 27-30, 2018, Proceedings, Part I
EditorsC. P. Rosé, R. Martínez-Maldonado, H. U. Hoppe, R. Luckin, M. Mavrikis, K. Porayska-Pomsta, B. McLaren, B. d. Boulay
Place of PublicationCham
PublisherSpringer International Publishing AG
Pages409-419
Number of pages11
ISBN (Electronic)9783319938431
ISBN (Print)9783319938424
DOIs
Publication statusPublished - 2018
Externally publishedYes
EventInternational Conference, AIED 2018 - London, United Kingdom
Duration: 27 Jun 201830 Jun 2018
https://aied2018.utscic.edu.au/

Publication series

SeriesLecture Notes in Computer Science
Volume10947
SeriesLecture Notes in Artificial Intelligence (subseries)

Conference

ConferenceInternational Conference, AIED 2018
Abbreviated titleAIED 2018
CountryUnited Kingdom
CityLondon
Period27/06/1830/06/18
Internet address

Keywords

  • Natural language processing
  • Intelligent tutoring systems
  • Self-explanations
  • Support vector machines
  • ReaderBench

Fingerprint Dive into the research topics of 'Bring It on! Challenges Encountered While Building a Comprehensive Tutoring System Using ReaderBench'. Together they form a unique fingerprint.

  • Projects

    Rage: Realising an Applied Gaming Eco-system

    Westera, W., Georgiadis, K., Saveski, G., van Lankveld, G., Bahreini, K., van der Vegt, W., Berkhout, J., Nyamsuren, E., Kluijfhout, E. & Nadolski, R.

    1/02/1531/07/19

    Project: Research

    Cite this

    Panaite, M., Dascalu, M., Johnson, A. M., Balyan, R., Dai, J., McNamara, D. S., & Trausan-Matu, S. (2018). Bring It on! Challenges Encountered While Building a Comprehensive Tutoring System Using ReaderBench. In C. P. Rosé, R. Martínez-Maldonado, H. U. Hoppe, R. Luckin, M. Mavrikis, K. Porayska-Pomsta, B. McLaren, & B. D. Boulay (Eds.), Artificial Intelligence in Education: 19th International Conference, AIED 2018, London, UK, June 27-30, 2018, Proceedings, Part I (pp. 409-419). Springer International Publishing AG. Lecture Notes in Computer Science, Vol.. 10947, Lecture Notes in Artificial Intelligence (subseries) https://doi.org/10.1007/978-3-319-93843-1_30