Bring It on! Challenges Encountered While Building a Comprehensive Tutoring System Using ReaderBench

Marilena Panaite, Mihai Dascalu, Amy M. Johnson, Renu Balyan, Jianmin Dai, Danielle S. McNamara, Stefan Trausan-Matu

Research output: Chapter in Book/Report/Conference proceedingConference Article in proceedingAcademicpeer-review

227 Downloads (Pure)


Intelligent Tutoring Systems (ITSs) are aimed at promoting acquisition of knowledge and skills by providing relevant and appropriate feedback during students’ practice activities. ITSs for literacy instruction commonly assess typed responses using Natural Language Processing (NLP) algorithms. One step in this direction often requires building a scoring mechanism that matches human judgments. This paper describes the challenges encountered while implementing an automated evaluation workflow and adopting solutions for increasing performance of the tutoring system. The algorithm described here comprises multiple stages, including initial pre-processing, a rule-based system for pre-classifying self-explanations, followed by classification using a Support Virtual Machine (SVM) learning algorithm. The SVM model hyper-parameters were optimized using grid search approach with 4,109 different self explanations scored 0 to 3 (i.e., poor to great). The accuracy achieved for the model was 59% (adjacent accuracy = 97%; Kappa = .43).
Original languageEnglish
Title of host publicationArtificial Intelligence in Education
Subtitle of host publication19th International Conference, AIED 2018, London, UK, June 27-30, 2018, Proceedings, Part I
EditorsManolis Mavrikis, Carolyn Penstein Rosé, Bruce McLaren, H. Ulrich Hoppe, Rose Luckin, Kaska Porayska-Pomsta, Benedict du Boulay, Roberto Martinez-Maldonado
Place of PublicationCham
PublisherSpringer International Publishing AG
Number of pages11
ISBN (Electronic)9783319938431
ISBN (Print)9783319938424
Publication statusPublished - 2018
Externally publishedYes
EventInternational Conference, AIED 2018 - London, United Kingdom
Duration: 27 Jun 201830 Jun 2018

Publication series

SeriesLecture Notes in Computer Science
SeriesLecture Notes in Artificial Intelligence (subseries)


ConferenceInternational Conference, AIED 2018
Abbreviated titleAIED 2018
Country/TerritoryUnited Kingdom
Internet address


  • Intelligent tutoring systems
  • Natural language processing
  • ReaderBench
  • Self-explanations
  • Support vector machines


Dive into the research topics of 'Bring It on! Challenges Encountered While Building a Comprehensive Tutoring System Using ReaderBench'. Together they form a unique fingerprint.

Cite this