Please ReaderBench This Text: A Multi-Dimensional Textual Complexity Assessment Framework

Mihai Dascalu, Scott A. Crossley, Danielle S. McNamara, Philippe Dessus, Stefan Trausan-Matu

    Research output: Chapter in Book/Report/Conference proceedingChapterAcademicpeer-review

    2 Downloads (Pure)

    Abstract

    A critical task for tutors is to provide learners with suitable reading materials in terms of difficulty. The challenge of this endeavor is increased by students’ individual variability and the multiple levels in which complexity can vary, thus arguing for the necessity of automated systems to support teachers. This chapter describes ReaderBench, an open-source multi-dimensional and multi-lingual system that uses advanced Natural Language Processing techniques to assess textual complexity at multiple levels including surface-based, syntax, semantics and discourse structure. In contrast to other existing approaches, ReaderBench is centered on cohesion and makes extensive usage of two complementary models, i.e., Cohesion Network Analysis and the polyphonic model inspired from dialogism. The first model provides an in-depth view of discourse in terms of cohesive links, whereas the second one highlights interactions between points of view spanning throughout the discourse. In order to argue for its wide applicability and extensibility, two studies are introduced. The first study investigates the degree to which ReaderBench textual complexity indices differentiate between high and low cohesion texts. The ReaderBench indices led to a higher classification accuracy than those included in prior studies using Coh-Metrix and TAACO. In the second study, ReaderBench indices are used to predict the difficulty of a set of various texts. Although the high number of predictive indices (50 plus) accounted for less variance than previous studies, they make valuable contributions to our understanding of text due to their wide coverage.
    Original languageEnglish
    Title of host publicationTutoring and Intelligent Tutoring Systems
    EditorsScotty D. Craig
    PublisherNova Science Publishers, Inc.
    Chapter9
    Pages251-271
    Number of pages21
    ISBN (Electronic)978-1-63485-211-1
    ISBN (Print)978-1-53614-085-9
    Publication statusPublished - 2018

    Fingerprint

    discourse
    network analysis
    tutor
    syntax
    coverage
    semantics
    teacher
    interaction
    language
    student

    Keywords

    • comprehension modeling
    • learning analytics
    • automated essay scoring
    • data analytics
    • Natural Language Processing

    Cite this

    Dascalu, M., Crossley, S. A., McNamara, D. S., Dessus, P., & Trausan-Matu, S. (2018). Please ReaderBench This Text: A Multi-Dimensional Textual Complexity Assessment Framework. In S. D. Craig (Ed.), Tutoring and Intelligent Tutoring Systems (pp. 251-271). Nova Science Publishers, Inc..
    Dascalu, Mihai ; Crossley, Scott A. ; McNamara, Danielle S. ; Dessus, Philippe ; Trausan-Matu, Stefan. / Please ReaderBench This Text : A Multi-Dimensional Textual Complexity Assessment Framework. Tutoring and Intelligent Tutoring Systems. editor / Scotty D. Craig . Nova Science Publishers, Inc., 2018. pp. 251-271
    @inbook{570d0214550f4b048d502eeb0e17d2d3,
    title = "Please ReaderBench This Text: A Multi-Dimensional Textual Complexity Assessment Framework",
    abstract = "A critical task for tutors is to provide learners with suitable reading materials in terms of difficulty. The challenge of this endeavor is increased by students’ individual variability and the multiple levels in which complexity can vary, thus arguing for the necessity of automated systems to support teachers. This chapter describes ReaderBench, an open-source multi-dimensional and multi-lingual system that uses advanced Natural Language Processing techniques to assess textual complexity at multiple levels including surface-based, syntax, semantics and discourse structure. In contrast to other existing approaches, ReaderBench is centered on cohesion and makes extensive usage of two complementary models, i.e., Cohesion Network Analysis and the polyphonic model inspired from dialogism. The first model provides an in-depth view of discourse in terms of cohesive links, whereas the second one highlights interactions between points of view spanning throughout the discourse. In order to argue for its wide applicability and extensibility, two studies are introduced. The first study investigates the degree to which ReaderBench textual complexity indices differentiate between high and low cohesion texts. The ReaderBench indices led to a higher classification accuracy than those included in prior studies using Coh-Metrix and TAACO. In the second study, ReaderBench indices are used to predict the difficulty of a set of various texts. Although the high number of predictive indices (50 plus) accounted for less variance than previous studies, they make valuable contributions to our understanding of text due to their wide coverage.",
    keywords = "comprehension modeling, learning analytics, automated essay scoring, data analytics, Natural Language Processing",
    author = "Mihai Dascalu and Crossley, {Scott A.} and McNamara, {Danielle S.} and Philippe Dessus and Stefan Trausan-Matu",
    year = "2018",
    language = "English",
    isbn = "978-1-53614-085-9",
    pages = "251--271",
    editor = "{Craig }, {Scotty D. }",
    booktitle = "Tutoring and Intelligent Tutoring Systems",
    publisher = "Nova Science Publishers, Inc.",
    address = "United States",

    }

    Dascalu, M, Crossley, SA, McNamara, DS, Dessus, P & Trausan-Matu, S 2018, Please ReaderBench This Text: A Multi-Dimensional Textual Complexity Assessment Framework. in SD Craig (ed.), Tutoring and Intelligent Tutoring Systems. Nova Science Publishers, Inc., pp. 251-271.

    Please ReaderBench This Text : A Multi-Dimensional Textual Complexity Assessment Framework. / Dascalu, Mihai; Crossley, Scott A. ; McNamara, Danielle S.; Dessus, Philippe; Trausan-Matu, Stefan.

    Tutoring and Intelligent Tutoring Systems. ed. / Scotty D. Craig . Nova Science Publishers, Inc., 2018. p. 251-271.

    Research output: Chapter in Book/Report/Conference proceedingChapterAcademicpeer-review

    TY - CHAP

    T1 - Please ReaderBench This Text

    T2 - A Multi-Dimensional Textual Complexity Assessment Framework

    AU - Dascalu, Mihai

    AU - Crossley, Scott A.

    AU - McNamara, Danielle S.

    AU - Dessus, Philippe

    AU - Trausan-Matu, Stefan

    PY - 2018

    Y1 - 2018

    N2 - A critical task for tutors is to provide learners with suitable reading materials in terms of difficulty. The challenge of this endeavor is increased by students’ individual variability and the multiple levels in which complexity can vary, thus arguing for the necessity of automated systems to support teachers. This chapter describes ReaderBench, an open-source multi-dimensional and multi-lingual system that uses advanced Natural Language Processing techniques to assess textual complexity at multiple levels including surface-based, syntax, semantics and discourse structure. In contrast to other existing approaches, ReaderBench is centered on cohesion and makes extensive usage of two complementary models, i.e., Cohesion Network Analysis and the polyphonic model inspired from dialogism. The first model provides an in-depth view of discourse in terms of cohesive links, whereas the second one highlights interactions between points of view spanning throughout the discourse. In order to argue for its wide applicability and extensibility, two studies are introduced. The first study investigates the degree to which ReaderBench textual complexity indices differentiate between high and low cohesion texts. The ReaderBench indices led to a higher classification accuracy than those included in prior studies using Coh-Metrix and TAACO. In the second study, ReaderBench indices are used to predict the difficulty of a set of various texts. Although the high number of predictive indices (50 plus) accounted for less variance than previous studies, they make valuable contributions to our understanding of text due to their wide coverage.

    AB - A critical task for tutors is to provide learners with suitable reading materials in terms of difficulty. The challenge of this endeavor is increased by students’ individual variability and the multiple levels in which complexity can vary, thus arguing for the necessity of automated systems to support teachers. This chapter describes ReaderBench, an open-source multi-dimensional and multi-lingual system that uses advanced Natural Language Processing techniques to assess textual complexity at multiple levels including surface-based, syntax, semantics and discourse structure. In contrast to other existing approaches, ReaderBench is centered on cohesion and makes extensive usage of two complementary models, i.e., Cohesion Network Analysis and the polyphonic model inspired from dialogism. The first model provides an in-depth view of discourse in terms of cohesive links, whereas the second one highlights interactions between points of view spanning throughout the discourse. In order to argue for its wide applicability and extensibility, two studies are introduced. The first study investigates the degree to which ReaderBench textual complexity indices differentiate between high and low cohesion texts. The ReaderBench indices led to a higher classification accuracy than those included in prior studies using Coh-Metrix and TAACO. In the second study, ReaderBench indices are used to predict the difficulty of a set of various texts. Although the high number of predictive indices (50 plus) accounted for less variance than previous studies, they make valuable contributions to our understanding of text due to their wide coverage.

    KW - comprehension modeling

    KW - learning analytics

    KW - automated essay scoring

    KW - data analytics

    KW - Natural Language Processing

    M3 - Chapter

    SN - 978-1-53614-085-9

    SP - 251

    EP - 271

    BT - Tutoring and Intelligent Tutoring Systems

    A2 - Craig , Scotty D.

    PB - Nova Science Publishers, Inc.

    ER -

    Dascalu M, Crossley SA, McNamara DS, Dessus P, Trausan-Matu S. Please ReaderBench This Text: A Multi-Dimensional Textual Complexity Assessment Framework. In Craig SD, editor, Tutoring and Intelligent Tutoring Systems. Nova Science Publishers, Inc. 2018. p. 251-271