Designing an Online Self-assessment for Informed Study Decisions: The User Perspective

L.E.C. Delnoij; J.P.W. Janssen; K.J.H. Dirkx; R.L. Martens

doi:10.1007/978-3-030-57717-9_6

Designing an Online Self-assessment for Informed Study Decisions: The User Perspective

L.E.C. Delnoij^*
, J.P.W. Janssen
, K.J.H. Dirkx
, R.L. Martens

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference Article in proceeding › Academic › peer-review

Abstract

This paper presents the results of a study, carried out as part of the design-based development of an online self-assessment for prospective students in higher online education. The self-assessment consists of a set of tests – predictive of completion – and is meant to improve informed decision making prior to enrolment. The rationale being that better decision making will help to address the ongoing concern of non-completion in higher online education. A prototypical design of the self-assessment was created based on an extensive literature review and correlational research, aimed at investigating validity evidence concerning the predictive value of the tests. The present study focused on investigating validity evidence regarding the content of the self-assessment (including the feedback it provides) from a user perspective. Results from a survey among prospective students (N = 66) indicated that predictive validity and content validity of the self-assessment are somewhat at odds: three out of the five tests included in the current prototype were considered relevant by prospective students. Moreover, students rated eleven additionally suggested tests – currently not included – as relevant concerning their study decision. Expectations regarding the feedback to be provided in connection with the tests include an explanation of the measurement and advice for further preparation. A comparison of the obtained scores to a reference group (i.e., other test-takers or successful students) is not expected. Implications for further development and evaluation of the self-assessment are discussed.

Original language	English
Title of host publication	Addressing Global Challenges and Quality Education
Subtitle of host publication	15th European Conference on Technology Enhanced Learning, EC-TEL 2020, Heidelberg, Germany, September 14–18, 2020, Proceedings
Editors	Carlos Alario-Hoyos, María Jesús Rodríguez-Triana, Maren Scheffel, Inmaculada Arnedillo-Sánchez, Sebastian Maximilian Dennerlein
Place of Publication	Cham
Publisher	Springer Nature Switzerland AG
Chapter	6
Pages	74-86
Number of pages	13
ISBN (Electronic)	9783030577179
ISBN (Print)	9783030577162
DOIs	https://doi.org/10.1007/978-3-030-57717-9_6
Publication status	Published - Sept 2020
Event	European Conference on Technology Enhanced Learning 2020: Addressing Global Challenges and Quality Education - Online, PH Heidelberg, Heidelberg, Germany Duration: 14 Sept 2020 → 18 Sept 2020 Conference number: 15 https://link.springer.com/chapter/10.1007%2F978-3-030-57717-9_46#citeas http://www.ec-tel.eu/index.php?id=957

Publication series

Series	Lecture Notes in Computer Science
Volume	12315
ISSN	0302-9743

Conference

Conference	European Conference on Technology Enhanced Learning 2020
Abbreviated title	ECTEL 2020
Country/Territory	Germany
City	Heidelberg
Period	14/09/20 → 18/09/20
Internet address	https://link.springer.com/chapter/10.1007%2F978-3-030-57717-9_46#citeas http://www.ec-tel.eu/index.php?id=957

Keywords

Design-based research
Feedback
Higher online education
Self-assessment
Study decision

Access to Document

10.1007/978-3-030-57717-9_6Licence: CC BY

Cite this

Delnoij, L. E. C., Janssen, J. P. W., Dirkx, K. J. H., & Martens, R. L. (2020). Designing an Online Self-assessment for Informed Study Decisions: The User Perspective. In C. Alario-Hoyos, M. J. Rodríguez-Triana, M. Scheffel, I. Arnedillo-Sánchez, & S. M. Dennerlein (Eds.), Addressing Global Challenges and Quality Education: 15th European Conference on Technology Enhanced Learning, EC-TEL 2020, Heidelberg, Germany, September 14–18, 2020, Proceedings (pp. 74-86). Springer Nature Switzerland AG. https://doi.org/10.1007/978-3-030-57717-9_6

Delnoij, L.E.C. ; Janssen, J.P.W. ; Dirkx, K.J.H. et al. / Designing an Online Self-assessment for Informed Study Decisions : The User Perspective. Addressing Global Challenges and Quality Education: 15th European Conference on Technology Enhanced Learning, EC-TEL 2020, Heidelberg, Germany, September 14–18, 2020, Proceedings. editor / Carlos Alario-Hoyos ; María Jesús Rodríguez-Triana ; Maren Scheffel ; Inmaculada Arnedillo-Sánchez ; Sebastian Maximilian Dennerlein. Cham : Springer Nature Switzerland AG, 2020. pp. 74-86 (Lecture Notes in Computer Science, Vol. 12315).

@inproceedings{d4e21210e74747bd91975cb2c00df3da,

title = "Designing an Online Self-assessment for Informed Study Decisions: The User Perspective",

abstract = "This paper presents the results of a study, carried out as part of the design-based development of an online self-assessment for prospective students in higher online education. The self-assessment consists of a set of tests {\textendash} predictive of completion {\textendash} and is meant to improve informed decision making prior to enrolment. The rationale being that better decision making will help to address the ongoing concern of non-completion in higher online education. A prototypical design of the self-assessment was created based on an extensive literature review and correlational research, aimed at investigating validity evidence concerning the predictive value of the tests. The present study focused on investigating validity evidence regarding the content of the self-assessment (including the feedback it provides) from a user perspective. Results from a survey among prospective students (N = 66) indicated that predictive validity and content validity of the self-assessment are somewhat at odds: three out of the five tests included in the current prototype were considered relevant by prospective students. Moreover, students rated eleven additionally suggested tests {\textendash} currently not included {\textendash} as relevant concerning their study decision. Expectations regarding the feedback to be provided in connection with the tests include an explanation of the measurement and advice for further preparation. A comparison of the obtained scores to a reference group (i.e., other test-takers or successful students) is not expected. Implications for further development and evaluation of the self-assessment are discussed.",

keywords = "Design-based research, Feedback, Higher online education, Self-assessment, Study decision",

author = "L.E.C. Delnoij and J.P.W. Janssen and K.J.H. Dirkx and R.L. Martens",

year = "2020",

month = sep,

doi = "10.1007/978-3-030-57717-9\_6",

language = "English",

isbn = "9783030577162",

series = "Lecture Notes in Computer Science",

publisher = "Springer Nature Switzerland AG",

pages = "74--86",

editor = "Carlos Alario-Hoyos and Rodr{\'i}guez-Triana, \{Mar{\'i}a Jes{\'u}s\} and Maren Scheffel and Inmaculada Arnedillo-S{\'a}nchez and Dennerlein, \{Sebastian Maximilian\}",

booktitle = "Addressing Global Challenges and Quality Education",

address = "Switzerland",

note = "European Conference on Technology Enhanced Learning 2020 : Addressing Global Challenges and Quality Education, ECTEL 2020 ; Conference date: 14-09-2020 Through 18-09-2020",

url = "https://link.springer.com/chapter/10.1007\%2F978-3-030-57717-9\_46\#citeas, http://www.ec-tel.eu/index.php?id=957",

}

Delnoij, LEC, Janssen, JPW, Dirkx, KJH & Martens, RL 2020, Designing an Online Self-assessment for Informed Study Decisions: The User Perspective. in C Alario-Hoyos, MJ Rodríguez-Triana, M Scheffel, I Arnedillo-Sánchez & SM Dennerlein (eds), Addressing Global Challenges and Quality Education: 15th European Conference on Technology Enhanced Learning, EC-TEL 2020, Heidelberg, Germany, September 14–18, 2020, Proceedings. Springer Nature Switzerland AG, Cham, Lecture Notes in Computer Science, vol. 12315, pp. 74-86, European Conference on Technology Enhanced Learning 2020, Heidelberg, Germany, 14/09/20. https://doi.org/10.1007/978-3-030-57717-9_6

Designing an Online Self-assessment for Informed Study Decisions: The User Perspective. / Delnoij, L.E.C.; Janssen, J.P.W.; Dirkx, K.J.H. et al.
Addressing Global Challenges and Quality Education: 15th European Conference on Technology Enhanced Learning, EC-TEL 2020, Heidelberg, Germany, September 14–18, 2020, Proceedings. ed. / Carlos Alario-Hoyos; María Jesús Rodríguez-Triana; Maren Scheffel; Inmaculada Arnedillo-Sánchez; Sebastian Maximilian Dennerlein. Cham: Springer Nature Switzerland AG, 2020. p. 74-86 (Lecture Notes in Computer Science, Vol. 12315).

Research output: Chapter in Book/Report/Conference proceeding › Conference Article in proceeding › Academic › peer-review

TY - GEN

T1 - Designing an Online Self-assessment for Informed Study Decisions

T2 - European Conference on Technology Enhanced Learning 2020

AU - Delnoij, L.E.C.

AU - Janssen, J.P.W.

AU - Dirkx, K.J.H.

AU - Martens, R.L.

N1 - Conference code: 15

PY - 2020/9

Y1 - 2020/9

N2 - This paper presents the results of a study, carried out as part of the design-based development of an online self-assessment for prospective students in higher online education. The self-assessment consists of a set of tests – predictive of completion – and is meant to improve informed decision making prior to enrolment. The rationale being that better decision making will help to address the ongoing concern of non-completion in higher online education. A prototypical design of the self-assessment was created based on an extensive literature review and correlational research, aimed at investigating validity evidence concerning the predictive value of the tests. The present study focused on investigating validity evidence regarding the content of the self-assessment (including the feedback it provides) from a user perspective. Results from a survey among prospective students (N = 66) indicated that predictive validity and content validity of the self-assessment are somewhat at odds: three out of the five tests included in the current prototype were considered relevant by prospective students. Moreover, students rated eleven additionally suggested tests – currently not included – as relevant concerning their study decision. Expectations regarding the feedback to be provided in connection with the tests include an explanation of the measurement and advice for further preparation. A comparison of the obtained scores to a reference group (i.e., other test-takers or successful students) is not expected. Implications for further development and evaluation of the self-assessment are discussed.

AB - This paper presents the results of a study, carried out as part of the design-based development of an online self-assessment for prospective students in higher online education. The self-assessment consists of a set of tests – predictive of completion – and is meant to improve informed decision making prior to enrolment. The rationale being that better decision making will help to address the ongoing concern of non-completion in higher online education. A prototypical design of the self-assessment was created based on an extensive literature review and correlational research, aimed at investigating validity evidence concerning the predictive value of the tests. The present study focused on investigating validity evidence regarding the content of the self-assessment (including the feedback it provides) from a user perspective. Results from a survey among prospective students (N = 66) indicated that predictive validity and content validity of the self-assessment are somewhat at odds: three out of the five tests included in the current prototype were considered relevant by prospective students. Moreover, students rated eleven additionally suggested tests – currently not included – as relevant concerning their study decision. Expectations regarding the feedback to be provided in connection with the tests include an explanation of the measurement and advice for further preparation. A comparison of the obtained scores to a reference group (i.e., other test-takers or successful students) is not expected. Implications for further development and evaluation of the self-assessment are discussed.

KW - Design-based research

KW - Feedback

KW - Higher online education

KW - Self-assessment

KW - Study decision

U2 - 10.1007/978-3-030-57717-9_6

DO - 10.1007/978-3-030-57717-9_6

M3 - Conference Article in proceeding

SN - 9783030577162

T3 - Lecture Notes in Computer Science

SP - 74

EP - 86

BT - Addressing Global Challenges and Quality Education

A2 - Alario-Hoyos, Carlos

A2 - Rodríguez-Triana, María Jesús

A2 - Scheffel, Maren

A2 - Arnedillo-Sánchez, Inmaculada

A2 - Dennerlein, Sebastian Maximilian

PB - Springer Nature Switzerland AG

CY - Cham

Y2 - 14 September 2020 through 18 September 2020

ER -

Delnoij LEC, Janssen JPW, Dirkx KJH, Martens RL. Designing an Online Self-assessment for Informed Study Decisions: The User Perspective. In Alario-Hoyos C, Rodríguez-Triana MJ, Scheffel M, Arnedillo-Sánchez I, Dennerlein SM, editors, Addressing Global Challenges and Quality Education: 15th European Conference on Technology Enhanced Learning, EC-TEL 2020, Heidelberg, Germany, September 14–18, 2020, Proceedings. Cham: Springer Nature Switzerland AG. 2020. p. 74-86. (Lecture Notes in Computer Science, Vol. 12315). doi: 10.1007/978-3-030-57717-9_6

Designing an Online Self-assessment for Informed Study Decisions: The User Perspective

Abstract

Publication series

Conference

Keywords

Access to Document

Fingerprint

Cite this