Toward an integrated analysis of verbal and visual data: The quest for expertise indicators

Medical expertise studies so far have mainly used verbal presentations of cases. These verbal presentations make that visual search and detection of symptoms are bypassed. Hence differences in that respect are not part of present medical expertise theories. Recently we have conducted a study in the domain of pathology, a visual domain par excellence, in which we compared medical students, residents and experienced pathologists (N=38) on seven cases. We collected diagnostic accuracy, think-aloud, eye-movement and navigation data (i.e., zooming and panning of digital slides), revealing (the interaction between) cognitive, perceptual and manipulatory processes. Earlier research suggests ways to analyse the separate data sets, but every data set also brings analysis problems on the table that have not been dealt with before as they are related to the features of this specific field. Examples are: How to identify encapsulated concepts in a visual domain? How to analyse eye tracking data on dynamic and manipulable images, more precisely areas of interest that change location and size on a manipulable image, or smooth pursuit eye movements following motion across the screen? And which indicators are most suitable to chart these processes? We expect, however, that the most interesting information will come from the integrated, time locked analysis of the three data sets. So far the medical domain does not provide examples to build on, but studies done in the linguistic domain may be helpful to develop strategies (e.g., Holsanova, 2008; Richardson & Dale, 2005). In this presentation example protocols will be presented, and the challenges and possible ways to a solution in the analyses will be discussed.