Identifying critical features for formative essay feedback with artificial neural networks and backward elimination

Mohsin Abbas; Peter van Rosmalen; Marco Kalz

doi:10.1007/978-3-030-29736-7_30

Identifying critical features for formative essay feedback with artificial neural networks and backward elimination

Mohsin Abbas^*, Peter van Rosmalen, Marco Kalz

^*Corresponding author for this work

Research Line Learning (part of LIRS program)

Research output: Chapter in Book/Report/Conference proceeding › Conference Article in proceeding › Academic › peer-review

Abstract

For predicting and improving the quality of essays, text analytic metrics (surface, syntactic, morphological and semantic features) can be used to provide formative feedback to the students. In this study, the intent was to find a small number of features that exhibit a fair proxy of the scores given by the human raters. Using an existing corpus and a text analysis tool for the Dutch language, a large number of features were extracted. Artificial neural networks, Levenberg Marquardt algorithm and backward elimination were used to reduce the number of extracted features automatically. Irrelevant features were eliminated based on the inter-rater agreement between predicted and human scores calculated using Cohen’s Kappa. By using our algorithm, the number of features in this study was reduced from 457 to 23. The selected features were grouped into six different categories. Of these categories, we believe that the features present in the groups “Word Difficulty” and “Lexical Diversity” are most useful for providing automated formative feedback to the students. The approach presented in this research paper is the first step towards our ultimate goal of providing meaningful formative feedback to the students for enhancing their writing skills and capabilities.

Original language	English
Title of host publication	Transforming Learning with Meaningful Technologies - 14th European Conference on Technology Enhanced Learning, EC-TEL 2019, Proceedings
Subtitle of host publication	14th European Conference on Technology Enhanced Learning, EC-TEL 2019, Delft, The Netherlands, September 16–19, 2019, Proceedings
Editors	Maren Scheffel, Julien Broisin, Viktoria Pammer-Schindler, Andri Ioannou, Jan Schneider
Place of Publication	Cham
Publisher	Springer
Chapter	30
Pages	396-408
Number of pages	13
ISBN (Electronic)	9783030297367
ISBN (Print)	9783030297350
DOIs	https://doi.org/10.1007/978-3-030-29736-7_30
Publication status	Published - 9 Sept 2019
Event	14th European Conference on Technology Enhanced Learning: Transforming Learning With Meaningful Technologies - The Leiden-Delft-Erasmus Center for Education and Learning , Delft, Netherlands Duration: 16 Sept 2019 → 19 Sept 2019 Conference number: 2019 http://www.ec-tel.eu http://www.ec-tel.eu/index.php?id=918

Publication series

Series	Lecture Notes in Computer Science
Volume	11722
ISSN	0302-9743

Conference

Conference	14th European Conference on Technology Enhanced Learning
Abbreviated title	EC-TEL 2019
Country/Territory	Netherlands
City	Delft
Period	16/09/19 → 19/09/19
Internet address	http://www.ec-tel.eu http://www.ec-tel.eu/index.php?id=918

Keywords

Backward Elimination
Dimensionality reduction
Feature selection
Formative feedback
Natural Language Processing
Neural Networks

Access to Document

10.1007/978-3-030-29736-7_30Licence: CC BY

Cite this

Abbas, M., van Rosmalen, P., & Kalz, M. (2019). Identifying critical features for formative essay feedback with artificial neural networks and backward elimination. In M. Scheffel, J. Broisin, V. Pammer-Schindler, A. Ioannou, & J. Schneider (Eds.), Transforming Learning with Meaningful Technologies - 14th European Conference on Technology Enhanced Learning, EC-TEL 2019, Proceedings: 14th European Conference on Technology Enhanced Learning, EC-TEL 2019, Delft, The Netherlands, September 16–19, 2019, Proceedings (pp. 396-408). Springer. https://doi.org/10.1007/978-3-030-29736-7_30

Abbas, Mohsin ; van Rosmalen, Peter ; Kalz, Marco. / Identifying critical features for formative essay feedback with artificial neural networks and backward elimination. Transforming Learning with Meaningful Technologies - 14th European Conference on Technology Enhanced Learning, EC-TEL 2019, Proceedings: 14th European Conference on Technology Enhanced Learning, EC-TEL 2019, Delft, The Netherlands, September 16–19, 2019, Proceedings. editor / Maren Scheffel ; Julien Broisin ; Viktoria Pammer-Schindler ; Andri Ioannou ; Jan Schneider. Cham : Springer, 2019. pp. 396-408 (Lecture Notes in Computer Science, Vol. 11722).

@inproceedings{5c53a8d22f32418284c94c1a575d6b17,

title = "Identifying critical features for formative essay feedback with artificial neural networks and backward elimination",

abstract = "For predicting and improving the quality of essays, text analytic metrics (surface, syntactic, morphological and semantic features) can be used to provide formative feedback to the students. In this study, the intent was to find a small number of features that exhibit a fair proxy of the scores given by the human raters. Using an existing corpus and a text analysis tool for the Dutch language, a large number of features were extracted. Artificial neural networks, Levenberg Marquardt algorithm and backward elimination were used to reduce the number of extracted features automatically. Irrelevant features were eliminated based on the inter-rater agreement between predicted and human scores calculated using Cohen{\textquoteright}s Kappa. By using our algorithm, the number of features in this study was reduced from 457 to 23. The selected features were grouped into six different categories. Of these categories, we believe that the features present in the groups “Word Difficulty” and “Lexical Diversity” are most useful for providing automated formative feedback to the students. The approach presented in this research paper is the first step towards our ultimate goal of providing meaningful formative feedback to the students for enhancing their writing skills and capabilities.",

keywords = "Backward Elimination, Dimensionality reduction, Feature selection, Formative feedback, Natural Language Processing, Neural Networks",

author = "Mohsin Abbas and {van Rosmalen}, Peter and Marco Kalz",

year = "2019",

month = sep,

day = "9",

doi = "10.1007/978-3-030-29736-7_30",

language = "English",

isbn = "9783030297350",

series = "Lecture Notes in Computer Science",

publisher = "Springer",

pages = "396--408",

editor = "Maren Scheffel and Julien Broisin and Viktoria Pammer-Schindler and Andri Ioannou and Jan Schneider",

booktitle = "Transforming Learning with Meaningful Technologies - 14th European Conference on Technology Enhanced Learning, EC-TEL 2019, Proceedings",

note = "14th European Conference on Technology Enhanced Learning : Transforming Learning With Meaningful Technologies, EC-TEL 2019 ; Conference date: 16-09-2019 Through 19-09-2019",

url = "http://www.ec-tel.eu, http://www.ec-tel.eu/index.php?id=918",

}

Abbas, M, van Rosmalen, P & Kalz, M 2019, Identifying critical features for formative essay feedback with artificial neural networks and backward elimination. in M Scheffel, J Broisin, V Pammer-Schindler, A Ioannou & J Schneider (eds), Transforming Learning with Meaningful Technologies - 14th European Conference on Technology Enhanced Learning, EC-TEL 2019, Proceedings: 14th European Conference on Technology Enhanced Learning, EC-TEL 2019, Delft, The Netherlands, September 16–19, 2019, Proceedings. Springer, Cham, Lecture Notes in Computer Science, vol. 11722, pp. 396-408, 14th European Conference on Technology Enhanced Learning, Delft, Netherlands, 16/09/19. https://doi.org/10.1007/978-3-030-29736-7_30

Identifying critical features for formative essay feedback with artificial neural networks and backward elimination. / Abbas, Mohsin; van Rosmalen, Peter; Kalz, Marco.
Transforming Learning with Meaningful Technologies - 14th European Conference on Technology Enhanced Learning, EC-TEL 2019, Proceedings: 14th European Conference on Technology Enhanced Learning, EC-TEL 2019, Delft, The Netherlands, September 16–19, 2019, Proceedings. ed. / Maren Scheffel; Julien Broisin; Viktoria Pammer-Schindler; Andri Ioannou; Jan Schneider. Cham: Springer, 2019. p. 396-408 (Lecture Notes in Computer Science, Vol. 11722).

Research output: Chapter in Book/Report/Conference proceeding › Conference Article in proceeding › Academic › peer-review

TY - GEN

T1 - Identifying critical features for formative essay feedback with artificial neural networks and backward elimination

AU - Abbas, Mohsin

AU - van Rosmalen, Peter

AU - Kalz, Marco

N1 - Conference code: 2019

PY - 2019/9/9

Y1 - 2019/9/9

N2 - For predicting and improving the quality of essays, text analytic metrics (surface, syntactic, morphological and semantic features) can be used to provide formative feedback to the students. In this study, the intent was to find a small number of features that exhibit a fair proxy of the scores given by the human raters. Using an existing corpus and a text analysis tool for the Dutch language, a large number of features were extracted. Artificial neural networks, Levenberg Marquardt algorithm and backward elimination were used to reduce the number of extracted features automatically. Irrelevant features were eliminated based on the inter-rater agreement between predicted and human scores calculated using Cohen’s Kappa. By using our algorithm, the number of features in this study was reduced from 457 to 23. The selected features were grouped into six different categories. Of these categories, we believe that the features present in the groups “Word Difficulty” and “Lexical Diversity” are most useful for providing automated formative feedback to the students. The approach presented in this research paper is the first step towards our ultimate goal of providing meaningful formative feedback to the students for enhancing their writing skills and capabilities.

AB - For predicting and improving the quality of essays, text analytic metrics (surface, syntactic, morphological and semantic features) can be used to provide formative feedback to the students. In this study, the intent was to find a small number of features that exhibit a fair proxy of the scores given by the human raters. Using an existing corpus and a text analysis tool for the Dutch language, a large number of features were extracted. Artificial neural networks, Levenberg Marquardt algorithm and backward elimination were used to reduce the number of extracted features automatically. Irrelevant features were eliminated based on the inter-rater agreement between predicted and human scores calculated using Cohen’s Kappa. By using our algorithm, the number of features in this study was reduced from 457 to 23. The selected features were grouped into six different categories. Of these categories, we believe that the features present in the groups “Word Difficulty” and “Lexical Diversity” are most useful for providing automated formative feedback to the students. The approach presented in this research paper is the first step towards our ultimate goal of providing meaningful formative feedback to the students for enhancing their writing skills and capabilities.

KW - Backward Elimination

KW - Dimensionality reduction

KW - Feature selection

KW - Formative feedback

KW - Natural Language Processing

KW - Neural Networks

U2 - 10.1007/978-3-030-29736-7_30

DO - 10.1007/978-3-030-29736-7_30

M3 - Conference Article in proceeding

SN - 9783030297350

T3 - Lecture Notes in Computer Science

SP - 396

EP - 408

BT - Transforming Learning with Meaningful Technologies - 14th European Conference on Technology Enhanced Learning, EC-TEL 2019, Proceedings

A2 - Scheffel, Maren

A2 - Broisin, Julien

A2 - Pammer-Schindler, Viktoria

A2 - Ioannou, Andri

A2 - Schneider, Jan

PB - Springer

CY - Cham

T2 - 14th European Conference on Technology Enhanced Learning

Y2 - 16 September 2019 through 19 September 2019

ER -

Abbas M, van Rosmalen P, Kalz M. Identifying critical features for formative essay feedback with artificial neural networks and backward elimination. In Scheffel M, Broisin J, Pammer-Schindler V, Ioannou A, Schneider J, editors, Transforming Learning with Meaningful Technologies - 14th European Conference on Technology Enhanced Learning, EC-TEL 2019, Proceedings: 14th European Conference on Technology Enhanced Learning, EC-TEL 2019, Delft, The Netherlands, September 16–19, 2019, Proceedings. Cham: Springer. 2019. p. 396-408. (Lecture Notes in Computer Science, Vol. 11722). doi: 10.1007/978-3-030-29736-7_30

Identifying critical features for formative essay feedback with artificial neural networks and backward elimination

Abstract

Publication series

Conference

Keywords

Access to Document

Fingerprint

Cite this