Co-speech gestures influence neural activity in brain regions associated with processing semantic information

Anthony Steven Dick, Susan Goldin-Meadow, Uri Hasson, Jeremy I. Skipper, Steven L. Small

Research output: Contribution to journalArticlepeer-review

125 Scopus citations

Abstract

Everyday communication is accompanied by visual information fromseveral sources, including cospeech gestures, which provide semantic information listeners use to help disambiguate the speaker's message. Using fMRI, we examined how gestures influence neural activity in brain regions associated with processing semantic information. The BOLD response was recorded while participants listened to stories under three audiovisual conditions and one auditory-only (speech alone) condition. In the first audiovisual condition, the storyteller produced gestures that naturally accompany speech. In the second, the storyteller made semantically unrelated hand movements. In the third, the storyteller kept her hands still. In addition to inferior parietal and posterior superior andmiddle temporal regions, bilateral posterior superior temporal sulcus and left anterior inferior frontal gyrus responded more strongly to speech when it was further accompanied by gesture, regardless of the semantic relation to speech. However, the right inferior frontal gyrus was sensitive to the semantic import of the hand movements, demonstrating more activity when hand movements were semantically unrelated to the accompanying speech. These findings show that perceiving hand movements during speech modulates the distributed pattern of neural activation involved in both biological motion perception and discourse comprehension, suggesting listeners attempt to find meaning, not only in the words speakers produce, but also in the hand movements that accompany speech.

Original languageEnglish (US)
Pages (from-to)3509-3526
Number of pages18
JournalHuman Brain Mapping
Volume30
Issue number11
DOIs
StatePublished - Nov 2009
Externally publishedYes

Keywords

  • Discourse comprehension
  • Gestures
  • Inferior frontal gyrus
  • Semantic processing
  • fMRI

ASJC Scopus subject areas

  • Anatomy
  • Radiological and Ultrasound Technology
  • Radiology Nuclear Medicine and imaging
  • Neurology
  • Clinical Neurology

Fingerprint

Dive into the research topics of 'Co-speech gestures influence neural activity in brain regions associated with processing semantic information'. Together they form a unique fingerprint.

Cite this