Towards an online self-assessment for informed study decisions: A mixed-methods validation study

Informed study decisions are pivotal for student retention in higher
online education. A self-assessment prior to enrolment has been proposed as a
promising approach to enable informed decision-making and to build resources for retention. To determine whether such a self-assessment affects the decision-making process as intended, thorough and careful validation is a necessity. This study reports on two validity aspects that are less commonly addressed in that respect, but essential for evaluating effectiveness: response processes and consequences of (self-) testing. To map the response processes and consequences of the current selfassessment, a mixed-methods approach was used in which eight prospective students took a self-assessment in an observed think-aloud mode and were interviewed before and after that. Results show different response processes depending on the type of subtest that is taken. The results also indicate that consequential aspect of validity must be considered in the context of decisionmaking phases. The demonstrated evidence and possible threats to validity are discussed in light of refining the self-assessment and embedding it in counselling practice.