Development and validation of a test for measuring primary school students' effective use of ICT: The ECC-ICT test

Background: A practical test that measures the information and communication technology(ICT) skills students need for effectively using ICT in primary education hasyet to be developed (Oh et al., 2021). This paper reports on the development, validation,and reliability of a test measuring primary school students' ICT skills required foreffectively using ICT (the ECC-ICT test).Objectives: Based on existing literature, three ICT use domains were identified foreffectively using ICT: Effective, collaborative, and creative use of ICT. For these threedomains, 24 corresponding teaching objectives were identified from a widely useddigital literacy framework. Thirty-four test items cover these teaching objectives inan online test.Methods: A mixed-method approach was used for the ECC-ICT test. Four pilotrounds (n=25) implemented qualitative interviews for cognitive validity and refining thetest items, followed by a qualitative usability study(n=6). Confirmatory factor analysis andANOVA provided quantitative insight into the large-scale test administration(n=575).Results and Conclusions: Composite reliability of our conceptual 3-factor confirmatorymodel showed that the test reliably measured primary school effective use ofICT (ω = 0.82), collaborative use of ICT (ω = 0.80) and creative use of ICT(ω = 0.64). Convergent validity (ranging from 0.41 to 0.46) was acceptable. Internalconsistency (ranging from 0.84 to 0.91) and discriminant validity (HTMT values below0.90) are good. ANOVA results show that mean test scores are higher for students inhigher grade levels (p < 0.001). The post hoc Bonferroni results show that mostgrade-by-grade comparisons are significant (p < 0.001).