An Evaluation Framework for Data Competitions in TEL

Hendrik Drachsler; Slavi Stoyanov; Mathieu d'Aquin; Eelco Herder; Marieke Guy; Stefan Dietze

doi:10.1007/978-3-319-11200-8_6

An Evaluation Framework for Data Competitions in TEL

Hendrik Drachsler, Slavi Stoyanov, Mathieu d'Aquin, Eelco Herder, Marieke Guy, Stefan Dietze

Research output: Chapter in Book/Report/Conference proceeding › Conference Article in proceeding › Academic › peer-review

Abstract

This paper presents a study describing the development of an Evaluation Framework (EF) for data competitions in TEL. The study applies the Group Concept Method (GCM) to empirically depict criteria and their indicators for evaluating software applications in TEL. A statistical analysis of multidimensional scaling and hierarchical clustering on the GCM data identified the following six evaluation criteria: 1.Educational Innovation, 2.Usability, 3.Data, 4.Performance, 5.Privacy, and 6.Audience. Each of them was operationalized through a set of indicators. The resulting Evaluation Framework (EF) including these criteria was applied to the first data competition of the LinkedUp project. The EF was consequently improved using the results from reviewers’ interviews, which were analysed qualitatively and quantitatively. The outcome of these efforts is a comprehensive EF that can be used for TEL data competitions and for the evaluation of TEL tools in general.

Original language	English
Title of host publication	Open Learning and Teaching in Educational Communities
Subtitle of host publication	9th European Conference on Technology Enhanced Learning, EC-TEL 2014, Graz, Austria, September 16-19, 2014, Proceedings
Editors	Christoph Rensing, Sara de Freitas, Tobias Ley, Pedro J. Muñoz-Merino
Place of Publication	Cham
Publisher	Springer
Pages	70-83
Number of pages	14
ISBN (Electronic)	978-3-319-11200-8
ISBN (Print)	978-3-319-11199-5
DOIs	https://doi.org/10.1007/978-3-319-11200-8_6
Publication status	Published - 2014
Event	9th European Conference on Technology Enhanced Learning: Open Learning and Teaching in Educational Communities - Graz, Austria Duration: 16 Sept 2014 → 19 Sept 2014 Conference number: 9 http://ectel2014.httc.de/index.php?id=681

Publication series

Series	Lecture Notes in Computer Science (LNCS)
Volume	8719
ISSN	0302-9743

Conference

Conference	9th European Conference on Technology Enhanced Learning
Abbreviated title	EC-TEL 2014
Country/Territory	Austria
City	Graz
Period	16/09/14 → 19/09/14
Internet address	http://ectel2014.httc.de/index.php?id=681

Keywords

data competition
Linked Data
evaluation framework

Access to Document

10.1007/978-3-319-11200-8_6

Cite this

Drachsler, H., Stoyanov, S., d'Aquin, M., Herder, E., Guy, M., & Dietze, S. (2014). An Evaluation Framework for Data Competitions in TEL. In C. Rensing, S. de Freitas, T. Ley, & P. J. Muñoz-Merino (Eds.), Open Learning and Teaching in Educational Communities: 9th European Conference on Technology Enhanced Learning, EC-TEL 2014, Graz, Austria, September 16-19, 2014, Proceedings (pp. 70-83). Springer. https://doi.org/10.1007/978-3-319-11200-8_6

Drachsler, Hendrik ; Stoyanov, Slavi ; d'Aquin, Mathieu et al. / An Evaluation Framework for Data Competitions in TEL. Open Learning and Teaching in Educational Communities: 9th European Conference on Technology Enhanced Learning, EC-TEL 2014, Graz, Austria, September 16-19, 2014, Proceedings. editor / Christoph Rensing ; Sara de Freitas ; Tobias Ley ; Pedro J. Muñoz-Merino. Cham : Springer, 2014. pp. 70-83 (Lecture Notes in Computer Science (LNCS), Vol. 8719).

@inproceedings{f681c58150374de8923b8759f5ef0761,

title = "An Evaluation Framework for Data Competitions in TEL",

abstract = "This paper presents a study describing the development of an Evaluation Framework (EF) for data competitions in TEL. The study applies the Group Concept Method (GCM) to empirically depict criteria and their indicators for evaluating software applications in TEL. A statistical analysis of multidimensional scaling and hierarchical clustering on the GCM data identified the following six evaluation criteria: 1.Educational Innovation, 2.Usability, 3.Data, 4.Performance, 5.Privacy, and 6.Audience. Each of them was operationalized through a set of indicators. The resulting Evaluation Framework (EF) including these criteria was applied to the first data competition of the LinkedUp project. The EF was consequently improved using the results from reviewers{\textquoteright} interviews, which were analysed qualitatively and quantitatively. The outcome of these efforts is a comprehensive EF that can be used for TEL data competitions and for the evaluation of TEL tools in general.",

keywords = "data competition, Linked Data, evaluation framework",

author = "Hendrik Drachsler and Slavi Stoyanov and Mathieu d'Aquin and Eelco Herder and Marieke Guy and Stefan Dietze",

year = "2014",

doi = "10.1007/978-3-319-11200-8_6",

language = "English",

isbn = "978-3-319-11199-5",

series = "Lecture Notes in Computer Science (LNCS)",

publisher = "Springer",

pages = "70--83",

editor = "Christoph Rensing and {de Freitas}, Sara and Tobias Ley and Mu{\~n}oz-Merino, {Pedro J.}",

booktitle = "Open Learning and Teaching in Educational Communities",

address = "United States",

note = "9th European Conference on Technology Enhanced Learning : Open Learning and Teaching in Educational Communities, EC-TEL 2014 ; Conference date: 16-09-2014 Through 19-09-2014",

url = "http://ectel2014.httc.de/index.php?id=681",

}

Drachsler, H , Stoyanov, S, d'Aquin, M, Herder, E, Guy, M & Dietze, S 2014, An Evaluation Framework for Data Competitions in TEL. in C Rensing, S de Freitas, T Ley & PJ Muñoz-Merino (eds), Open Learning and Teaching in Educational Communities: 9th European Conference on Technology Enhanced Learning, EC-TEL 2014, Graz, Austria, September 16-19, 2014, Proceedings. Springer, Cham, Lecture Notes in Computer Science (LNCS), vol. 8719, pp. 70-83, 9th European Conference on Technology Enhanced Learning, Graz, Austria, 16/09/14. https://doi.org/10.1007/978-3-319-11200-8_6

An Evaluation Framework for Data Competitions in TEL. / Drachsler, Hendrik ; Stoyanov, Slavi; d'Aquin, Mathieu et al.
Open Learning and Teaching in Educational Communities: 9th European Conference on Technology Enhanced Learning, EC-TEL 2014, Graz, Austria, September 16-19, 2014, Proceedings. ed. / Christoph Rensing; Sara de Freitas; Tobias Ley; Pedro J. Muñoz-Merino. Cham: Springer, 2014. p. 70-83 (Lecture Notes in Computer Science (LNCS), Vol. 8719).

Research output: Chapter in Book/Report/Conference proceeding › Conference Article in proceeding › Academic › peer-review

TY - GEN

T1 - An Evaluation Framework for Data Competitions in TEL

AU - Drachsler, Hendrik

AU - Stoyanov, Slavi

AU - d'Aquin, Mathieu

AU - Herder, Eelco

AU - Guy, Marieke

AU - Dietze, Stefan

N1 - Conference code: 9

PY - 2014

Y1 - 2014

N2 - This paper presents a study describing the development of an Evaluation Framework (EF) for data competitions in TEL. The study applies the Group Concept Method (GCM) to empirically depict criteria and their indicators for evaluating software applications in TEL. A statistical analysis of multidimensional scaling and hierarchical clustering on the GCM data identified the following six evaluation criteria: 1.Educational Innovation, 2.Usability, 3.Data, 4.Performance, 5.Privacy, and 6.Audience. Each of them was operationalized through a set of indicators. The resulting Evaluation Framework (EF) including these criteria was applied to the first data competition of the LinkedUp project. The EF was consequently improved using the results from reviewers’ interviews, which were analysed qualitatively and quantitatively. The outcome of these efforts is a comprehensive EF that can be used for TEL data competitions and for the evaluation of TEL tools in general.

AB - This paper presents a study describing the development of an Evaluation Framework (EF) for data competitions in TEL. The study applies the Group Concept Method (GCM) to empirically depict criteria and their indicators for evaluating software applications in TEL. A statistical analysis of multidimensional scaling and hierarchical clustering on the GCM data identified the following six evaluation criteria: 1.Educational Innovation, 2.Usability, 3.Data, 4.Performance, 5.Privacy, and 6.Audience. Each of them was operationalized through a set of indicators. The resulting Evaluation Framework (EF) including these criteria was applied to the first data competition of the LinkedUp project. The EF was consequently improved using the results from reviewers’ interviews, which were analysed qualitatively and quantitatively. The outcome of these efforts is a comprehensive EF that can be used for TEL data competitions and for the evaluation of TEL tools in general.

KW - data competition

KW - Linked Data

KW - evaluation framework

U2 - 10.1007/978-3-319-11200-8_6

DO - 10.1007/978-3-319-11200-8_6

M3 - Conference Article in proceeding

SN - 978-3-319-11199-5

T3 - Lecture Notes in Computer Science (LNCS)

SP - 70

EP - 83

BT - Open Learning and Teaching in Educational Communities

A2 - Rensing, Christoph

A2 - de Freitas, Sara

A2 - Ley, Tobias

A2 - Muñoz-Merino, Pedro J.

PB - Springer

CY - Cham

T2 - 9th European Conference on Technology Enhanced Learning

Y2 - 16 September 2014 through 19 September 2014

ER -

Drachsler H , Stoyanov S, d'Aquin M, Herder E, Guy M, Dietze S. An Evaluation Framework for Data Competitions in TEL. In Rensing C, de Freitas S, Ley T, Muñoz-Merino PJ, editors, Open Learning and Teaching in Educational Communities: 9th European Conference on Technology Enhanced Learning, EC-TEL 2014, Graz, Austria, September 16-19, 2014, Proceedings. Cham: Springer. 2014. p. 70-83. (Lecture Notes in Computer Science (LNCS), Vol. 8719). doi: 10.1007/978-3-319-11200-8_6

An Evaluation Framework for Data Competitions in TEL

Abstract

Publication series

Conference

Keywords

Access to Document

Fingerprint

Cite this