Universität Bielefeld - Sonderforschungsbereich 360

Eye Tracking Data and the Retention of Gestural Information

Marianne Gullberg
Lund University, Dept. of Linguistics
Lund, Sweden

The so-called mismatch studies have shown that listeners retain information originally expressed in the gestural modality (Cassell, McNeill & McCullough, forthcoming). It can be assumed that the retention of gestural information is mediated through visual attention. Listeners usually look at the speaker's face, however, and only a minority of the speaker's gestures are visually fixated (Gullberg & Holmqvist, forthcoming).

Using an eye tracker to study fixations in face-to-face communication, we found a smaller proportion of gestures to be fixated than the proportion of retained gestures reported in Cassell et al..

While fixations are overt physiological events, both attention and the integration of gestural information into the listener's cognitive representation are cognitive phenomena. The relationship between these three concepts is fundamentally important to the evaluation and use of fixation data, and to the interpretation of the results in the mismatch studies.

References

Cassell, J., McNeill, D., and McCullough, K.-E. Forthcoming.
"Speech-gesture mismatches: Evidence for one underlying representation of linguistic and nonlinguistic information". Pragmatics and Cognition.
Gullberg, M., and Holmqvist, K. Forthcoming.
"Keeping an eye on gestures: Visual perception of gestures in face-to-face communication". Pragmatics and Cognition.

Anke Weinberger, 1998-11-09