Source evaluation of domain experts and novices during Web search

Saskia Brand-Gruwel*, Yvonne Kammerer, Ludo van Meeuwen, Tamara van Gog

*Corresponding author for this work

Research output: Contribution to journalArticleAcademicpeer-review

Abstract

Nowadays, almost everyone uses the World Wide Web (WWW) to search for information of any kind. In education, students frequently use the WWW for selecting information to accomplish assignments such as writing an essay or preparing a presentation. The evaluation of sources and information is an important sub-skill in this process. But many students have not yet optimally developed this skill. On the basis of verbal reports, eye-tracking data, and navigation logs this study investigated how novices in the domain of psychology evaluate Internet sources as compared to domain experts. In addition, two different verbal reporting techniques, namely thinking aloud and cued retrospective reporting, were compared in order to examine students' evaluation behavior. Results revealed that domain expertise has an impact on individuals' evaluation behavior during Web search, such that domain experts showed a more sophisticated use of evaluation criteria to judge the reliability of sources and information and selected more reliable information than domain novices. Furthermore, the different verbal reporting techniques did not lead to different conclusions on criteria use in relation to domain expertise, although in general more utterances concerning evaluation of sources and information were expressed during cued retrospective reporting.
Original languageEnglish
Pages (from-to)234-251
Number of pages18
JournalJournal of Computer Assisted Learning
Volume33
Issue number3
DOIs
Publication statusPublished - Jun 2017

Fingerprint

expert
behavior evaluation
evaluation
Internet
expertise
student
psychology
education

Keywords

  • source evaluation
  • web search

Cite this

@article{328a4491d5ed4054ac2efc4aa62990d0,
title = "Source evaluation of domain experts and novices during Web search",
abstract = "Nowadays, almost everyone uses the World Wide Web (WWW) to search for information of any kind. In education, students frequently use the WWW for selecting information to accomplish assignments such as writing an essay or preparing a presentation. The evaluation of sources and information is an important sub-skill in this process. But many students have not yet optimally developed this skill. On the basis of verbal reports, eye-tracking data, and navigation logs this study investigated how novices in the domain of psychology evaluate Internet sources as compared to domain experts. In addition, two different verbal reporting techniques, namely thinking aloud and cued retrospective reporting, were compared in order to examine students' evaluation behavior. Results revealed that domain expertise has an impact on individuals' evaluation behavior during Web search, such that domain experts showed a more sophisticated use of evaluation criteria to judge the reliability of sources and information and selected more reliable information than domain novices. Furthermore, the different verbal reporting techniques did not lead to different conclusions on criteria use in relation to domain expertise, although in general more utterances concerning evaluation of sources and information were expressed during cued retrospective reporting.",
keywords = "source evaluation, web search",
author = "Saskia Brand-Gruwel and Yvonne Kammerer and {van Meeuwen}, Ludo and {van Gog}, Tamara",
note = "DS_Citation:Brand-Gruwel, S., Kammerer, Y., Van Meeuwen, L., & Van Gog (2017). Source evaluation of domain experts and novices during Web search. Journal of Computer Assisted Learning, 33(3), 234-251.",
year = "2017",
month = "6",
doi = "10.1111/jcal.12162",
language = "English",
volume = "33",
pages = "234--251",
journal = "Journal of Computer Assisted Learning",
issn = "0266-4909",
publisher = "Wiley",
number = "3",

}

Source evaluation of domain experts and novices during Web search. / Brand-Gruwel, Saskia; Kammerer, Yvonne; van Meeuwen, Ludo; van Gog, Tamara.

In: Journal of Computer Assisted Learning, Vol. 33, No. 3, 06.2017, p. 234-251.

Research output: Contribution to journalArticleAcademicpeer-review

TY - JOUR

T1 - Source evaluation of domain experts and novices during Web search

AU - Brand-Gruwel, Saskia

AU - Kammerer, Yvonne

AU - van Meeuwen, Ludo

AU - van Gog, Tamara

N1 - DS_Citation:Brand-Gruwel, S., Kammerer, Y., Van Meeuwen, L., & Van Gog (2017). Source evaluation of domain experts and novices during Web search. Journal of Computer Assisted Learning, 33(3), 234-251.

PY - 2017/6

Y1 - 2017/6

N2 - Nowadays, almost everyone uses the World Wide Web (WWW) to search for information of any kind. In education, students frequently use the WWW for selecting information to accomplish assignments such as writing an essay or preparing a presentation. The evaluation of sources and information is an important sub-skill in this process. But many students have not yet optimally developed this skill. On the basis of verbal reports, eye-tracking data, and navigation logs this study investigated how novices in the domain of psychology evaluate Internet sources as compared to domain experts. In addition, two different verbal reporting techniques, namely thinking aloud and cued retrospective reporting, were compared in order to examine students' evaluation behavior. Results revealed that domain expertise has an impact on individuals' evaluation behavior during Web search, such that domain experts showed a more sophisticated use of evaluation criteria to judge the reliability of sources and information and selected more reliable information than domain novices. Furthermore, the different verbal reporting techniques did not lead to different conclusions on criteria use in relation to domain expertise, although in general more utterances concerning evaluation of sources and information were expressed during cued retrospective reporting.

AB - Nowadays, almost everyone uses the World Wide Web (WWW) to search for information of any kind. In education, students frequently use the WWW for selecting information to accomplish assignments such as writing an essay or preparing a presentation. The evaluation of sources and information is an important sub-skill in this process. But many students have not yet optimally developed this skill. On the basis of verbal reports, eye-tracking data, and navigation logs this study investigated how novices in the domain of psychology evaluate Internet sources as compared to domain experts. In addition, two different verbal reporting techniques, namely thinking aloud and cued retrospective reporting, were compared in order to examine students' evaluation behavior. Results revealed that domain expertise has an impact on individuals' evaluation behavior during Web search, such that domain experts showed a more sophisticated use of evaluation criteria to judge the reliability of sources and information and selected more reliable information than domain novices. Furthermore, the different verbal reporting techniques did not lead to different conclusions on criteria use in relation to domain expertise, although in general more utterances concerning evaluation of sources and information were expressed during cued retrospective reporting.

KW - source evaluation

KW - web search

U2 - 10.1111/jcal.12162

DO - 10.1111/jcal.12162

M3 - Article

VL - 33

SP - 234

EP - 251

JO - Journal of Computer Assisted Learning

JF - Journal of Computer Assisted Learning

SN - 0266-4909

IS - 3

ER -