Frontal and temporal contributions to understanding the iconic co-speech gestures that accompany speech

Anthony Steven Dick, Eva H. Mok, Anjali Raja Beharelle, Susan Goldin-Meadow, Steven L. Small

Research output: Contribution to journalArticle

41 Citations (Scopus)

Abstract

In everyday conversation, listeners often rely on a speaker's gestures to clarify any ambiguities in the verbal message. Using fMRI during naturalistic story comprehension, we examined which brain regions in the listener are sensitive to speakers' iconic gestures. We focused on iconic gestures that contribute information not found in the speaker's talk, compared with those that convey information redundant with the speaker's talk. We found that three regions-left inferior frontal gyrus triangular (IFGTr) and opercular (IFGOp) portions, and left posterior middle temporal gyrus (MTGp)-responded more strongly when gestures added information to nonspecific language, compared with when they conveyed the same information in more specific language; in other words, when gesture disambiguated speech as opposed to reinforced it. An increased BOLD response was not found in these regions when the nonspecific language was produced without gesture, suggesting that IFGTr, IFGOp, and MTGp are involved in integrating semantic information across gesture and speech. In addition, we found that activity in the posterior superior temporal sulcus (STSp), previously thought to be involved in gesture-speech integration, was not sensitive to the gesture-speech relation. Together, these findings clarify the neurobiology of gesture-speech integration and contribute to an emerging picture of how listeners glean meaning from gestures that accompany speech.

Original languageEnglish (US)
Pages (from-to)900-917
Number of pages18
JournalHuman Brain Mapping
Volume35
Issue number3
DOIs
StatePublished - Mar 1 2014
Externally publishedYes

Fingerprint

Gestures
Language
Temporal Lobe
Prefrontal Cortex
Neurobiology
Semantics
Magnetic Resonance Imaging

Keywords

  • Gestures
  • Inferior frontal gyrus
  • Language
  • Posterior middle temporal gyrus
  • Posterior superior temporal sulcus
  • Semantic

ASJC Scopus subject areas

  • Anatomy
  • Radiological and Ultrasound Technology
  • Radiology Nuclear Medicine and imaging
  • Neurology
  • Clinical Neurology

Cite this

Frontal and temporal contributions to understanding the iconic co-speech gestures that accompany speech. / Dick, Anthony Steven; Mok, Eva H.; Beharelle, Anjali Raja; Goldin-Meadow, Susan; Small, Steven L.

In: Human Brain Mapping, Vol. 35, No. 3, 01.03.2014, p. 900-917.

Research output: Contribution to journalArticle

Dick, Anthony Steven ; Mok, Eva H. ; Beharelle, Anjali Raja ; Goldin-Meadow, Susan ; Small, Steven L. / Frontal and temporal contributions to understanding the iconic co-speech gestures that accompany speech. In: Human Brain Mapping. 2014 ; Vol. 35, No. 3. pp. 900-917.
@article{f978e75936dd4086a22dc4a60750c462,
title = "Frontal and temporal contributions to understanding the iconic co-speech gestures that accompany speech",
abstract = "In everyday conversation, listeners often rely on a speaker's gestures to clarify any ambiguities in the verbal message. Using fMRI during naturalistic story comprehension, we examined which brain regions in the listener are sensitive to speakers' iconic gestures. We focused on iconic gestures that contribute information not found in the speaker's talk, compared with those that convey information redundant with the speaker's talk. We found that three regions-left inferior frontal gyrus triangular (IFGTr) and opercular (IFGOp) portions, and left posterior middle temporal gyrus (MTGp)-responded more strongly when gestures added information to nonspecific language, compared with when they conveyed the same information in more specific language; in other words, when gesture disambiguated speech as opposed to reinforced it. An increased BOLD response was not found in these regions when the nonspecific language was produced without gesture, suggesting that IFGTr, IFGOp, and MTGp are involved in integrating semantic information across gesture and speech. In addition, we found that activity in the posterior superior temporal sulcus (STSp), previously thought to be involved in gesture-speech integration, was not sensitive to the gesture-speech relation. Together, these findings clarify the neurobiology of gesture-speech integration and contribute to an emerging picture of how listeners glean meaning from gestures that accompany speech.",
keywords = "Gestures, Inferior frontal gyrus, Language, Posterior middle temporal gyrus, Posterior superior temporal sulcus, Semantic",
author = "Dick, {Anthony Steven} and Mok, {Eva H.} and Beharelle, {Anjali Raja} and Susan Goldin-Meadow and Small, {Steven L.}",
year = "2014",
month = "3",
day = "1",
doi = "10.1002/hbm.22222",
language = "English (US)",
volume = "35",
pages = "900--917",
journal = "Human Brain Mapping",
issn = "1065-9471",
publisher = "Wiley-Liss Inc.",
number = "3",

}

TY - JOUR

T1 - Frontal and temporal contributions to understanding the iconic co-speech gestures that accompany speech

AU - Dick, Anthony Steven

AU - Mok, Eva H.

AU - Beharelle, Anjali Raja

AU - Goldin-Meadow, Susan

AU - Small, Steven L.

PY - 2014/3/1

Y1 - 2014/3/1

N2 - In everyday conversation, listeners often rely on a speaker's gestures to clarify any ambiguities in the verbal message. Using fMRI during naturalistic story comprehension, we examined which brain regions in the listener are sensitive to speakers' iconic gestures. We focused on iconic gestures that contribute information not found in the speaker's talk, compared with those that convey information redundant with the speaker's talk. We found that three regions-left inferior frontal gyrus triangular (IFGTr) and opercular (IFGOp) portions, and left posterior middle temporal gyrus (MTGp)-responded more strongly when gestures added information to nonspecific language, compared with when they conveyed the same information in more specific language; in other words, when gesture disambiguated speech as opposed to reinforced it. An increased BOLD response was not found in these regions when the nonspecific language was produced without gesture, suggesting that IFGTr, IFGOp, and MTGp are involved in integrating semantic information across gesture and speech. In addition, we found that activity in the posterior superior temporal sulcus (STSp), previously thought to be involved in gesture-speech integration, was not sensitive to the gesture-speech relation. Together, these findings clarify the neurobiology of gesture-speech integration and contribute to an emerging picture of how listeners glean meaning from gestures that accompany speech.

AB - In everyday conversation, listeners often rely on a speaker's gestures to clarify any ambiguities in the verbal message. Using fMRI during naturalistic story comprehension, we examined which brain regions in the listener are sensitive to speakers' iconic gestures. We focused on iconic gestures that contribute information not found in the speaker's talk, compared with those that convey information redundant with the speaker's talk. We found that three regions-left inferior frontal gyrus triangular (IFGTr) and opercular (IFGOp) portions, and left posterior middle temporal gyrus (MTGp)-responded more strongly when gestures added information to nonspecific language, compared with when they conveyed the same information in more specific language; in other words, when gesture disambiguated speech as opposed to reinforced it. An increased BOLD response was not found in these regions when the nonspecific language was produced without gesture, suggesting that IFGTr, IFGOp, and MTGp are involved in integrating semantic information across gesture and speech. In addition, we found that activity in the posterior superior temporal sulcus (STSp), previously thought to be involved in gesture-speech integration, was not sensitive to the gesture-speech relation. Together, these findings clarify the neurobiology of gesture-speech integration and contribute to an emerging picture of how listeners glean meaning from gestures that accompany speech.

KW - Gestures

KW - Inferior frontal gyrus

KW - Language

KW - Posterior middle temporal gyrus

KW - Posterior superior temporal sulcus

KW - Semantic

UR - http://www.scopus.com/inward/record.url?scp=84893814922&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84893814922&partnerID=8YFLogxK

U2 - 10.1002/hbm.22222

DO - 10.1002/hbm.22222

M3 - Article

C2 - 23238964

AN - SCOPUS:84893814922

VL - 35

SP - 900

EP - 917

JO - Human Brain Mapping

JF - Human Brain Mapping

SN - 1065-9471

IS - 3

ER -