Affective values in human decision-making 

Get Complete Project Material File(s) Now! »

Prefrontal cortex neuroanatomy, cytoarchitecture and neuro- physiology

The frontal lobes are particularly developed in humans compared to other species. They form a third of the brain surface and correspond to its most anterior part, incorporating both hemispheres. In this section, we present prefrontal cortex subdivisions according to anatomical landmarks. Functional subdivisions will be discussed in the next section.
Prefrontal cortex is delineated caudally by motor cortex. Premotor cortex and supple-mentary motor area (SMA) are usually not considered part of prefrontal cortex. Gyri and sulci, giving the human brain its characteristic folded appearance, constitute anatomical landmarks to decompose prefrontal cortex into distinct subparts. However, a decompo-sition based on Brodmann areas, which is not inconsistent with gyri and sulci, is more often used.
Brodmann areas (BA) give subregions delineation given cytoarchitecture, which refers to the cellular properties of the neural networks composing the di erent cortical layers. This classi cation is therefore based on the apparent structural organization of the cortex: number and thickness of cortical layers, dendritic arborization, etc. Figure 1.2 display the Brodmann areas composing prefrontal cortex.
The lateral part of prefrontal cortex comprehends, from rostral to caudal: BA 47 (OFC, ventrally); BA 10 (frontal pole, the most anterior part); BA 46 and BA 9 (roughly corresponding to dorsolateral PFC); BA 8, including frontal eye eld (Figure 1.2). On the left hemisphere, BA 44 and BA 45 (inferior frontal gyrus) correspond to Broca area, an area necessary for speech production, that is triggered during semantic tasks, semantic working memory and retrieval, as well as phonological and syntactic process-ing. The medial part comprises BA 24 (ventral anterior cingulate), BA 25 (subgenus, governing amygdala, insula and hippocampus), BA 32 (dorsal anterior cingulate) and BA 33 (pregenual cingulate). Here, I would like to emphasize the importance of these anatomical landmarks to study functionality. Indeed, usually computational models and neuroimaging do not take into account anatomical bases to elucidate brain subregions’ function.
Neurotransmitters and connectivity. Cortical layers are composed of excitatory and inhibitory neurons, that have long distance reciprocal projections with the rest of the cortex. Prefrontal cortex presents high intrinsic connectivity, as well as extrinsic con-nectivity with other brain regions. All neuromodulators types are present in prefrontal cortex (Fuster, 1988 [7]). Speci cally, dopamine and norepinephrine, thought to mediate learning (Collins and Frank, 2012 [8]), are found in higher concentrations than in other brain regions. Prefrontal cortex also has glutamatergic projections to the limbic system, e.g. amygdala and hippocampus, which are modulating emotional and memory-related responses, as well as neurons projecting to the thalamus and hypothalamus. Mutual connections i.e. that feed the PFC and that the PFC feeds involve sensory areas and posterior associative areas, making prefrontal cortex a center of convergence for various sensory inputs.
The connections pattern was originally investigated using tracers injection in non-human primates (Petrides and Pandya, 2002 [9]). Today, di usion tensor imaging (DTI) allowed to uncover part of these tracks (Croxson et al., 2005 [10]) and dress parallels with non-human primates functional regionalization.

Prefrontal cortex in non-human primates and other species

We will see in the next chapters that a lot of what we know about brain structure and function derive from the contribution of animal studies. Although my PhD work con-cerns the human brain, this section is a complement concerning animal brain anatomy.
Non-human primates brain share homologies with human brain regions (Wise et al., 2008 [11]), as shown in Figure 1.3. In rats, the homology of structures with human pre-frontal cortex is still debated, however the spatial distribution of cortical layers suggests homologies between rodents and primates (regarding granular areas, up to layer IV). Re-garding OFC, neural activity and connectivity is largely shared between rats, primates and humans (Preuss, 1995 [12]). Despite a smaller size for OFC in rats, causing less ability to handle complex cognitive tasks as compared to primates, lesions in this area lead to the same dysfunction pattern across species in tasks with reversal learning and with reward devaluation (Stalnaker et al., 2015 [13]).
Medial area and cingulate sulcus share equivalent functional homologies between humans and macaque monkeys (Procyk et al., 2014 [14]). More precisely, the term anterior cin-gulate cortex (ACC) corresponds to di erent parts of the mid-cingulate sulcus according to di erent studies. Within the cingulate sulcus, we can distinguish the most anterior part (ACC), followed by the midcingulate cortex (MCC) also referred to as dorsal ACC (dACC). Furthermore, certain human subjects have an additional cingulate sulcus which is dorsal to the rst one, named paracingulate sulcus (Petrides et al., 2012 [15]; Amiez et al., 2013 [16]).
In monkeys, the cingulate cortex presents similar cytoarchitectonic subparts, except for BA 32 which seems speci c to humans (Vogt, 2009a [17]). Also, there is no paracingulate sulcus in monkeys. However, the most anterior part, corresponding to BA10, remains more developed in humans and seems to comprise cognitive processes that are speci c to humans. Macaques contribution to neuroscience research involves local eld potential (LFP) and unitary extracellular electrophysiology recordings, in behaving animal. How-ever, this approach is not restricted to animals anymore. Recent studies start to use the same type of electrophysiology recordings in humans, with epileptic patients implanted with intracranial electrodes (Bonini, Burle et al., 2014 [18]).

Prefrontal cortex development and evolution during lifetime

In humans, prefrontal cortex is the last brain area to mature. Its development starts early in fetal life, in parallel with sensory and motor regions development, but is not over at birth and keeps growing during childhood and adolescence, up to 20 years old. The prefrontal endogenous circuits, driven by sensory stimulations, develop mostly during prenatal life, while the \cognitive » circuit appears at 7-12 months old. The maximal number of synapses and the complete maturation of certain cortical layers take place during the rst few years of life (Gazzaniga, Chapter 2, 2009 [19]). At that moment, the number of synapses is much higher than in adults. The presence of extra-synapses allows to selectively stabilize certain functional circuits more than others, in response to various environmental stimuli and experiences, through intense pruning of supernu-merary synapses. Synaptic connectivity exhibits initial exuberant production followed by gradual pruning (4-6 years old), with synapses density decreasing. The adult brain is then much less plastic.
Our faculty of judgment and decision is thus not complete until the prefrontal cortex is fully set up. Myelinization is not over until the second decade of life. Its development particularly depends on the amount and nature of exposure to stimuli, particularly to social stimuli, that are often complex and ambiguous. Blakemore’s team has shown that prefrontal cortex in relation with social cognition keeps developing until late adolescence (Blakemore, 2010 [20]). These changes in behavior and cognitive skills are accompanied by changes in brain structure and in grey matter volume, regarding for example medial prefrontal cortex (Blakemore, 2008 [21]). Sense of self and relational reasoning also expand during adolescence (Dumontheil et al., 2010 [22]). Thus, executive function, which underlie our faculty of judgment and our sense of responsibility is not complete until the age of 18-20 years old. In the next section, we will now describe diseases arising as a consequence of PFC dysfunction.

Neuropsychiatric diseases involving prefrontal cortex dysfunction

Besides vascular lesions, many neuropsychiatric disorders in humans involve speci c prefrontal de cits.
Obsessive-compulsive disorder. Patients with obsessive-compulsive disorder (OCD) display dysfunctional activity in orbitofrontal cortex, causing less behavioral exibility (Chamberlain et al. 2008 [23]), as well as abnormal fronto-striatal loops functioning. OCD also involves basal ganglia dysfunction, leading to compulsive and repetitive be-haviors (Baxter et al., 1992 [24]).
Addiction. Original addiction studies have focused on the reward circuit de cits in subcortical regions, such as ventral tegmental area. However, a growing body of ev-idence, coming from neuroimaging studies, revealed a key involvement of prefrontal cortex in drug addiction (Goldstein and Volkow, 2011 [25]), with an abnormal cognitive, motivational and emotional functions regulation. These studies indicated a decrease in cognitive control and in self-control in general, and a decrease of the ability to in-hibit drives, characterized by a self-awareness lowering in intoxication periods (Baler and Volkow, 2006 [26]). Speci cally, orbitofrontal and anterior cingulate cortices dys-function implies over-saliency of stimuli related to addiction and under-saliency of other reinforcers.
Schizophrenia. In schizophrenia, post-mortem studies revealed a reduced brain vol-ume, in particular in PFC and hippocampus, accompanied with abnormal cellular size, dendritic density and neural distribution. At the cellular and molecular levels, schizophrenic brain exhibits abnormal synaptic pruning during adolescence and early adulthood, corresponding to the symptoms onset. At the cognitive level, perceptual decision-making in schizophrenic patients is characterized by an over-dependence on prior expectations, despite sensory evidence being in contradiction with their prior ex-pectations (Blackwood et al., 2001 [27]). This tendency to base decision on less evidence than healthy subjects has been termed \jump-to-conclusion » bias (Moritz et al., 2005 [28]). More precisely, Jardri and Deneve proposed a hierarchical neural network explain-ing circular belief propagation (Jardri and Deneve, 2013 [29]). This circular belief prop-agation results in abnormal interaction between top-down and bottom-up information (Fletcher and Frith, 2008 [30]). Their model explained schizophrenic patients’ in exible beliefs (Woodward et al., 2008 [31]) as well as their overcon dence in front of probabilis-tic choices. Moreover, the over-reliance on prior expectations hypothesis is supported by several data sets (Barbalat, Rouault et al., 2012 [32]; Chambon et al., 2011 [33]). Lastly, Barbalat and colleagues tested schizophrenic participants in a task involving top-down cognitive control and maintenance of information from past events. Participants with schizophrenia had increased episodic control but had impaired contextual control (Bar-balat et al., 2009 [34]). In addition, schizophrenic patients were impaired in e ective connectivity within di erent lateral prefrontal cortex subparts, leading to a top-down control disconnection (Barbalat et al., 2011 [35]).
Historically, studying patients have provided some insight about the PFC functional roles. We will review in the next section the main theories of PFC functional architecture.

Functional and anatomical organization: main theories of prefrontal cortex function

In this section, we describe the proposed functions for the principal subregions of human prefrontal cortex. Roughly, human prefrontal cortex is organized around three main axes (Figure 1.4):
Motivational control (medial)
Cognitive
control
(dorsal)
Emotional
control
(ventral)
Figure 1.4: Prefrontal cortex and action control (coronal slice).
Motivational control, which refers to drives, underlying voluntary action. Cognitive control, which refers to rules and choices.
Emotional control, which refers to preferences.
Here, the term \control » refers to processes that are not automatic but controlled re-sponses. First, we will see that dorsolateral prefrontal cortex (Figure 1.5) is responsible for top-down cognitive action control, while ventrolateral prefrontal cortex is related to motor inhibition and updating action plans. Next, we will see that ventromedial and orbitofrontal cortices (Figure 1.5) are heterogeneous brain regions, involved in particu-lar in the outcomes and goods valuation, and in the values and emotions integration.
The following part will be dedicated to dorsomedial and cingulate cortices, implicated in motivation and performance monitoring. Finally, we will discuss the most in uential accounts proposed to underlie frontopolar cortex, the most anterior part of the human brain.
Figure 1.5: Main anatomical subdivisions within prefrontal cortex (reproduced from Szczepanski and Knight, 2014).

Lateral prefrontal cortex and hierarchical cognitive control

Lateral prefrontal cortex is implicated in goal-directed behavior. As such, it imple-ments the behavioral adjustments that the medial PFC indicates, maintaining represen-tations despite interference from distractors or irrelevant events until a goad is achieved. Lateral PFC is able to inhibit spontaneous responses before a motor action is executed. Lateral PFC is more activated following error trials, providing evidence for an increase in cognitive control for subsequent trials. As such, lateral PFC implements cognitive control adjustments.
Koechlin and colleagues have demonstrated a hierarchy in cognitive control within lateral prefrontal cortex, according to the information level. Here information is understood in the sense of Shannon information theory. At the sensory level, control is implemented in lateral premotor cortex, to select responses to stimuli (Koechlin et al., 2003 [36]). Certain neurons in lateral premotor encode planning an impeding movement and motor preparation. At the contextual level, caudal lateral PFC regions subserve control, in relation to external contextual cues associated with stimuli. Critically, contextual control is only engaged when current task contingencies require it (Collins and Frank, 2013 [37]). Finally, episodic control is implemented in rostral lateral PFC areas, given past behavioral episodes or internal goals, controlling more caudal regions in a \cascade » model. Top-down control is thus implemented according to a hierarchy in information processing and map onto a hierarchy in functional brain regions (Figure 1.6).
Figure 1.6: The cascade model of top-down cognitive control within lateral PFC (reproduced from Koechlin et al., 2003).
Rules implementation. In line with the notion of cognitive control, lateral PFC is in-volved in rule-based normative behavior. For example, Ru and colleagues were able to increase or decrease compliance to social normative rules in humans, when manipulating right lateral PFC using transcranial direct current stimulation (tDCS) (Ru et al., 2013 [38]). Behavioral rules neural substrates are found in lateral PFC in match-to-sample tasks. Ventrolateral PFC maintains rule representations to implement rule-based be-havior (Sakai and Passingham, 2003 [39], 2006 [40]). Lateral PFC is not involved in encoding simple stimulus/reward rules but is necessary to encode more abstract high level rules and behavioral strategies (Bunge et al., 2005 [41]; Genovesio et al., 2005 [42]).
Working memory. The working memory concept refers to the ability to maintain relevant information to perform a task at hand, in the short-term, at a more abstract level than sensory information processing. This temporary information maintenance allows learning, comprehension and reasoning (Baddeley, 2010 [43]). Working memory is more about function than about contents. It includes an attentional focus mechanism, which consists of a bottleneck, meaning that only a limited amount of information can be handled at the same time (Oberauer, 2002 [44]; Oberauer and Kliegl, 2006 [45]). This mechanism allows to select sets or representations by determining priorities between various informations. Working memory function is critically dependent on lateral PFC (Levy and Goldman-Rakic, 1999 [46]), for instance for overcoming interfering stimuli.
In summary, lateral PFC is highly specialized regarding its anatomy and function. It has an integrative and adaptive role in a range of executive control behaviors, including retention, information manipulation and retrieval to achieve long-term goals, via action planning (Fuster, 2001 [47]), as well as response inhibition and rules implementation.
Ventrolateral PFC is involved in active information retrieval and selection, whereas dorsolateral PFC is rather involved in very controlled processes.

Table of contents :

1 Human Decision-Making and Prefrontal Function 
1.1 Prefrontal cortex subserves central executive function
1.1.1 Early insights into prefrontal cortex functions: the contribution of lesion studies
1.1.2 Prefrontal cortex neuroanatomy, cytoarchitecture and neurophysiology
1.1.3 Prefrontal cortex in non-human primates and other species
1.1.4 Prefrontal cortex development and evolution during lifetime
1.1.5 Neuropsychiatric diseases involving prefrontal cortex dysfunction
1.2 Functional and anatomical organization: main theories of prefrontal cortex function
1.2.1 Lateral prefrontal cortex and hierarchical cognitive control
1.2.2 Ventromedial prefrontal cortex and orbitofrontal cortex
1.2.3 Dorsomedial prefrontal cortex and cingulate cortex
1.2.4 Frontopolar cortex
1.2.5 Conclusion
2 Affective values in human decision-making 
2.1 Aective values: psychological and theoretical aspects
2.1.1 The notion of affective value, based on rewards and punishments
2.1.2 Expected utility theory and prospect theory
2.1.3 Rewards are driving learning: pavlovian and instrumental conditioning
2.1.4 Reinforcement learning computational models
2.1.5 Model-based and model-free reinforcement learning
2.2 Aective values: cerebral aspects
2.2.1 Basal ganglia
2.2.1.1 Subcortical basal ganglia anatomy
2.2.1.2 Electrophysiology and pharmacology studies show reward prediction error in dopamine neurons
2.2.1.3 The contribution of neuroimaging studies
2.2.2 Medial prefrontal cortex
2.2.2.1 Pain and punishments neural correlates
2.2.3 Conclusion
3 Inferences in human decision-making 
3.1 Inferential processes: psychological and theoretical aspects
3.1.1 Probabilistic models of learning and reasoning
3.1.2 Bayesian inference
3.1.3 Possible limits of the Bayesian approach
3.1.4 Application of Bayesian inference models to learning and decisionmaking
3.2 Inferential processes: cerebral aspects
3.2.1 Model-based neuro-imaging
3.2.2 A role for vmPFC in inference
3.2.3 The medial PFC functional architecture in decision-making
4 Research question 
5 Protocol A: Decorrelate affective value from information of outcomes 
5.1 Experiment 1
5.1.1 Experimental design
5.1.2 Randomization
5.1.3 Experiment presentation
5.1.4 Participants
5.1.5 Statistical analysis
5.1.6 Computational modeling
5.1.6.1 Reinforcement learning model
5.1.6.2 Bayesian inference model
5.1.6.3 Bayesian inference model with online learning
5.1.6.4 Decay model
5.1.6.5 Mixed model
5.1.6.6 Action selection
5.1.7 Fitting procedure
5.1.7.1 Model selection
5.1.7.2 Quantitative measures
5.1.7.3 Qualitative measures
5.2 Experiment 2
5.2.1 Experimental design
5.2.2 Randomization, Experiment presentation and Participants
5.2.3 Statistical analysis and modeling
5.3 Experiment 3
5.3.1 Experimental design
5.3.2 Statistical analysis and modeling
6 Protocol A: Results and Discussion 
6.1 Experiment 1
6.1.1 Experiment 1: Behavioral Results
6.1.2 Experiment 1: Modeling Results
6.1.3 Experiment 1: Discussion
6.2 Experiment 2
6.2.1 Experiment 2: Behavioral Results
6.2.2 Experiment 2: Modeling Results
6.2.3 Experiment 2: Discussion
6.3 Experiment 3
6.3.1 Experiment 3: Behavioral Results
6.3.2 Experiment 3: Modeling Results
6.3.3 Experiment 3: Discussion
6.4 Experiments 1, 2 and 3: Conclusion
7 Protocol B: Integration of beliefs and affective values in decision- making 
7.1 Probabilistic reversal-learning task
7.1.1 Paradigm
7.1.2 Design and Randomization
7.1.3 Trial Structure and Jittering
7.1.4 Participants
7.1.5 Training
7.1.6 Debrieng
7.2 Behavioral Analyses
7.2.1 Learning Curves
7.2.2 Logistic Regressions
7.3 Computational Modeling
7.3.1 First class of models
7.3.2 Second class of models
7.3.3 Third class of models
7.3.4 Action selection
7.3.5 Fitting procedure
7.3.6 Model selection
7.3.6.1 Quantitative measures
7.3.6.2 Qualitative measures
7.4 Neuroimaging
7.4.1 fMRI acquisition
7.4.2 fMRI pre-processing
7.4.3 fMRI: Model-based approach
7.4.4 fMRI: General Linear Model
7.4.4.1 GLM1: Decision Values
7.4.4.2 GLM2: Dissociation belief system/affective values system
7.4.4.3 GLM3: Further dissociation within each system
7.4.5 Regions of Interest (ROI)
7.4.6 3D Bins analysis
8 Protocol B: Results 
8.1 Behavior
8.1.1 Learning curves
8.1.2 Rewards
8.1.3 Logistic regressions
8.1.4 Stay/Switch trials
8.1.5 Reaction times
8.2 Modeling
8.2.1 First class of models
8.2.2 Second class of models
8.2.3 Third class of models
8.2.4 Model selection
8.2.5 Best-tting mixed model parameters
8.2.6 Informational Values
8.2.7 Conclusion
8.3 Neuroimaging
8.3.1 GLM1: Decision Values
8.3.1.1 Choice-dependent eects
8.3.1.2 Choice-independent eects
8.3.2 GLM2: Dissociation belief system/aective values system
8.3.2.1 Choice-dependent eects
8.3.2.2 Choice-independent eects
8.3.3 Replication of results in an independent analysis
8.3.4 GLM3: Further dissociation within each system
8.3.4.1 Dissociation within the aective values system: Reinforcement values (historical) vs. Affective values of proposed rewards (current trial)
8.3.4.2 Dissociation within the Bayesian system: Prior belief (historical) vs. Informational values (current trial)
8.3.5 Activations at feedback time
9 General Discussion 
9.1 Modeling
9.1.1 Distortions
9.1.1.1 Differences with prospect theory
9.1.1.2 Sub-optimality and effecient coding
9.1.2 Predominance of the belief system
9.1.3 Interaction between the belief system and the affective value system: a hierarchy?
9.1.4 Difference with model-based/model-free reinforcement learning
9.1.5 Prefrontal cortex: a not yet optimized system?
9.1.6 Beliefs, affective values, and stability of representations
9.2 Imaging results: understanding the role of vmPFC and MCC
9.2.1 vmPFC and reliability signals
9.2.2 vmPFC and the default mode network
9.2.3 vmPFC and the notion of value
9.2.4 vmPFC and the notion of confidence
9.2.5 MCC and the affective values representation
9.3 General Conclusion
A Informal debrieng for thefirst series of behavioral experiments
B Instructions for fMRI experiment
C Informal debrieng following the last fMRI session
D Generative model of the fMRI task
D.1 Task Description
D.2 Bayesian inference, known parameters
D.2.1 Inference
D.2.2 Action selection
D.2.3 Contributions to action selection
D.3 Reinforcement learning
D.3.1 Inference
D.3.2 Action selection
D.3.3 Contributions to action selection
Bibliography 
List of Figures
List of Tables

GET THE COMPLETE PROJECT