Human reversal learning under conditions of certain versus uncertain outcomes

Anna Maria D'Cruz, Michael E. Ragozzino, Matthew W. Mosconi, Mani N. Pavuluri, John A. Sweeney

Research output: Contribution to journalArticle

27 Citations (Scopus)

Abstract

Reversal learning tasks assess behavioral flexibility by requiring subjects to switch from one learned response choice to a different response choice when task contingencies change. This requires both the processing of negative feedback once a learned response is no longer reinforced, and the capacity for flexible response selection. In 2-choice reversal learning tasks, subjects switch between only two responses. Multiple choice reversal learning is qualitatively different in that at reversal, it requires subjects to respond to non-reinforcement of a learned response by selecting a new response from among several alternatives that have uncertain consequences. While activity in brain regions responsible for processing unexpected negative feedback is known to increase in relation to the hedonic value of the reward itself, it is not known whether the uncertainty of reinforcement for future response choices also modulates these responses. In an fMRI study, 15 participants performed 2- and 4-choice reversal learning tasks. Upon reversal in both tasks, activation was observed in brain regions associated with processing changing reinforcement contingencies (midbrain, ventral striatum, insula), as well as in neocortical regions that support cognitive control and behavioral planning (prefrontal, premotor, posterior parietal, and anterior cingulate cortices). Activation in both systems was greater in the 4- than in the 2-choice task. Therefore, reinforcement uncertainty for future responses enhanced activity in brain systems that process performance feedback, as well as in areas supporting behavioral planning of future response choices. A mutually facilitative integration of responses in motivational and cognitive brain systems might enhance behavioral flexibility and decision making in conditions for which outcomes for future response choices are uncertain.

Original languageEnglish (US)
Pages (from-to)315-322
Number of pages8
JournalNeuroImage
Volume56
Issue number1
DOIs
StatePublished - May 1 2011

Fingerprint

Reversal Learning
Gyrus Cinguli
Brain
Uncertainty
Pleasure
Mesencephalon
Reward
Decision Making
Magnetic Resonance Imaging
Reinforcement (Psychology)

Keywords

  • Cognitive flexibility
  • FMRI
  • Human
  • Reinforcement learning
  • Reversal learning
  • Reward processing

ASJC Scopus subject areas

  • Cognitive Neuroscience
  • Neurology

Cite this

D'Cruz, A. M., Ragozzino, M. E., Mosconi, M. W., Pavuluri, M. N., & Sweeney, J. A. (2011). Human reversal learning under conditions of certain versus uncertain outcomes. NeuroImage, 56(1), 315-322. https://doi.org/10.1016/j.neuroimage.2011.01.068

Human reversal learning under conditions of certain versus uncertain outcomes. / D'Cruz, Anna Maria; Ragozzino, Michael E.; Mosconi, Matthew W.; Pavuluri, Mani N.; Sweeney, John A.

In: NeuroImage, Vol. 56, No. 1, 01.05.2011, p. 315-322.

Research output: Contribution to journalArticle

D'Cruz, AM, Ragozzino, ME, Mosconi, MW, Pavuluri, MN & Sweeney, JA 2011, 'Human reversal learning under conditions of certain versus uncertain outcomes', NeuroImage, vol. 56, no. 1, pp. 315-322. https://doi.org/10.1016/j.neuroimage.2011.01.068
D'Cruz AM, Ragozzino ME, Mosconi MW, Pavuluri MN, Sweeney JA. Human reversal learning under conditions of certain versus uncertain outcomes. NeuroImage. 2011 May 1;56(1):315-322. https://doi.org/10.1016/j.neuroimage.2011.01.068
D'Cruz, Anna Maria ; Ragozzino, Michael E. ; Mosconi, Matthew W. ; Pavuluri, Mani N. ; Sweeney, John A. / Human reversal learning under conditions of certain versus uncertain outcomes. In: NeuroImage. 2011 ; Vol. 56, No. 1. pp. 315-322.
@article{745dbee341ed4cefaa997d818793f4d3,
title = "Human reversal learning under conditions of certain versus uncertain outcomes",
abstract = "Reversal learning tasks assess behavioral flexibility by requiring subjects to switch from one learned response choice to a different response choice when task contingencies change. This requires both the processing of negative feedback once a learned response is no longer reinforced, and the capacity for flexible response selection. In 2-choice reversal learning tasks, subjects switch between only two responses. Multiple choice reversal learning is qualitatively different in that at reversal, it requires subjects to respond to non-reinforcement of a learned response by selecting a new response from among several alternatives that have uncertain consequences. While activity in brain regions responsible for processing unexpected negative feedback is known to increase in relation to the hedonic value of the reward itself, it is not known whether the uncertainty of reinforcement for future response choices also modulates these responses. In an fMRI study, 15 participants performed 2- and 4-choice reversal learning tasks. Upon reversal in both tasks, activation was observed in brain regions associated with processing changing reinforcement contingencies (midbrain, ventral striatum, insula), as well as in neocortical regions that support cognitive control and behavioral planning (prefrontal, premotor, posterior parietal, and anterior cingulate cortices). Activation in both systems was greater in the 4- than in the 2-choice task. Therefore, reinforcement uncertainty for future responses enhanced activity in brain systems that process performance feedback, as well as in areas supporting behavioral planning of future response choices. A mutually facilitative integration of responses in motivational and cognitive brain systems might enhance behavioral flexibility and decision making in conditions for which outcomes for future response choices are uncertain.",
keywords = "Cognitive flexibility, FMRI, Human, Reinforcement learning, Reversal learning, Reward processing",
author = "D'Cruz, {Anna Maria} and Ragozzino, {Michael E.} and Mosconi, {Matthew W.} and Pavuluri, {Mani N.} and Sweeney, {John A.}",
year = "2011",
month = "5",
day = "1",
doi = "10.1016/j.neuroimage.2011.01.068",
language = "English (US)",
volume = "56",
pages = "315--322",
journal = "NeuroImage",
issn = "1053-8119",
publisher = "Academic Press Inc.",
number = "1",

}

TY - JOUR

T1 - Human reversal learning under conditions of certain versus uncertain outcomes

AU - D'Cruz, Anna Maria

AU - Ragozzino, Michael E.

AU - Mosconi, Matthew W.

AU - Pavuluri, Mani N.

AU - Sweeney, John A.

PY - 2011/5/1

Y1 - 2011/5/1

N2 - Reversal learning tasks assess behavioral flexibility by requiring subjects to switch from one learned response choice to a different response choice when task contingencies change. This requires both the processing of negative feedback once a learned response is no longer reinforced, and the capacity for flexible response selection. In 2-choice reversal learning tasks, subjects switch between only two responses. Multiple choice reversal learning is qualitatively different in that at reversal, it requires subjects to respond to non-reinforcement of a learned response by selecting a new response from among several alternatives that have uncertain consequences. While activity in brain regions responsible for processing unexpected negative feedback is known to increase in relation to the hedonic value of the reward itself, it is not known whether the uncertainty of reinforcement for future response choices also modulates these responses. In an fMRI study, 15 participants performed 2- and 4-choice reversal learning tasks. Upon reversal in both tasks, activation was observed in brain regions associated with processing changing reinforcement contingencies (midbrain, ventral striatum, insula), as well as in neocortical regions that support cognitive control and behavioral planning (prefrontal, premotor, posterior parietal, and anterior cingulate cortices). Activation in both systems was greater in the 4- than in the 2-choice task. Therefore, reinforcement uncertainty for future responses enhanced activity in brain systems that process performance feedback, as well as in areas supporting behavioral planning of future response choices. A mutually facilitative integration of responses in motivational and cognitive brain systems might enhance behavioral flexibility and decision making in conditions for which outcomes for future response choices are uncertain.

AB - Reversal learning tasks assess behavioral flexibility by requiring subjects to switch from one learned response choice to a different response choice when task contingencies change. This requires both the processing of negative feedback once a learned response is no longer reinforced, and the capacity for flexible response selection. In 2-choice reversal learning tasks, subjects switch between only two responses. Multiple choice reversal learning is qualitatively different in that at reversal, it requires subjects to respond to non-reinforcement of a learned response by selecting a new response from among several alternatives that have uncertain consequences. While activity in brain regions responsible for processing unexpected negative feedback is known to increase in relation to the hedonic value of the reward itself, it is not known whether the uncertainty of reinforcement for future response choices also modulates these responses. In an fMRI study, 15 participants performed 2- and 4-choice reversal learning tasks. Upon reversal in both tasks, activation was observed in brain regions associated with processing changing reinforcement contingencies (midbrain, ventral striatum, insula), as well as in neocortical regions that support cognitive control and behavioral planning (prefrontal, premotor, posterior parietal, and anterior cingulate cortices). Activation in both systems was greater in the 4- than in the 2-choice task. Therefore, reinforcement uncertainty for future responses enhanced activity in brain systems that process performance feedback, as well as in areas supporting behavioral planning of future response choices. A mutually facilitative integration of responses in motivational and cognitive brain systems might enhance behavioral flexibility and decision making in conditions for which outcomes for future response choices are uncertain.

KW - Cognitive flexibility

KW - FMRI

KW - Human

KW - Reinforcement learning

KW - Reversal learning

KW - Reward processing

UR - http://www.scopus.com/inward/record.url?scp=79953049199&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=79953049199&partnerID=8YFLogxK

U2 - 10.1016/j.neuroimage.2011.01.068

DO - 10.1016/j.neuroimage.2011.01.068

M3 - Article

C2 - 21281720

AN - SCOPUS:79953049199

VL - 56

SP - 315

EP - 322

JO - NeuroImage

JF - NeuroImage

SN - 1053-8119

IS - 1

ER -