Universität Bielefeld - Sonderforschungsbereich 360
Eye Tracking Data and the Retention of Gestural Information
Marianne Gullberg
Lund University, Dept. of Linguistics
Lund, Sweden
The so-called mismatch studies have shown that listeners retain information originally expressed
in the gestural modality (Cassell, McNeill & McCullough, forthcoming). It can be assumed that the
retention of gestural information is mediated through visual attention. Listeners usually look at the
speaker's face, however, and only a minority of the speaker's gestures are visually fixated
(Gullberg & Holmqvist, forthcoming).
Using an eye tracker to study fixations in face-to-face communication, we found a smaller
proportion of gestures to be fixated than the proportion of retained gestures reported in Cassell et
al..
While fixations are overt physiological events, both attention and the integration of gestural
information into the listener's cognitive representation are cognitive phenomena. The relationship
between these three concepts is fundamentally important to the evaluation and use of fixation data,
and to the interpretation of the results in the mismatch studies.
References
- Cassell, J., McNeill, D., and McCullough, K.-E. Forthcoming.
- "Speech-gesture mismatches: Evidence for one underlying representation of
linguistic and nonlinguistic information". Pragmatics and Cognition.
- Gullberg, M., and Holmqvist, K. Forthcoming.
- "Keeping an eye on gestures: Visual perception of
gestures in face-to-face communication". Pragmatics and Cognition.
Anke Weinberger, 1998-11-09