Project Topics
Get Complete Project Material File(s) Now! »
Interactions between the valuation systems
Those systems are supposed to work in parallel and to assign different values to the same options, sometimes leading to a conflict (or a competition) between them. For example, Wayne Hershberger (Hershberger, 1986) showed that chicks were not able to learn to run away from food to get access to it. This example is an illustration of the failure of the instrumental system against the Pavlovian one (Dayan et al., 2006). Thus, the co-existence of these systems might (partially) explain why we often observe inconsistencies in behavior and choices.
A more recent view opposes the habitual system to the goal-directed system in a distinction between model-free and model-based valuations (Daw et al., 2005) to explain discrepancies in choices. The main idea underlying this framework is that the low cost model-free system computes value of options only on a reinforcement learning mode (only based on the previous output of the choice) while the model-based system would be more computationally demanding since it supposes that values are assigned in accordance to a learned model of the world. The knowledge of the environment and the internal states are comprised in this system which would correspond to the goal-directed system. This view allows explaining unexpected behaviors and supposes that the two systems can compete to assign values to options. Moreover, it has been shown that a mixture of those two systems is used by subjects to solve tasks designed to test those systems (see Daw et al., 2005).
In conclusion, values can be assigned with different systems but when we focus on binary choice tasks which require no learning, we can suppose that the goal-directed system is mainly involved, especially in the context of the laboratory (in which there is no habit). However, before entering the process of decision in this kind of task, we need to describe some properties of subjective values.
Subjective value properties
Many times, it has been observed than human is not rational, i.e. his choices cannot be predicted at 100% even if the subjective values of considered options have been reported. Numerous of violations of rationality have been described and it has been shown that values assigned to options can be modulated by factors such as uncertainty, risk, time or context. Interactions between values and these variables have been well described and formalized by economists in order to explain inconsistencies and biases in choices and why human is not purely rational. The next part addresses the impact of various variables on subjective valuation.
Diminishing marginal utility
As we saw in the historical section of value investigation, there is a nonlinear relationship between the amount of reward and the subjective value (Bernoulli, 1954). This phenomenon has been described as the ‘diminishing marginal utility’ and imposes a concave utility function to the amount of reward (Figure 2). Thus, we can define the utility as: = ( ).
With U being the Utility, or subjective value associated to the reward R and f a concave function. This is classically explaining non-linearity observed between amounts of reward and subjective values.
External environment
The impact of external environment is mainly seen in choices and violations of rationality seem to increase when the number of options available in the environment increases (DeShazo and Fermo, 2002). This can be explained by a cost to evaluate a lot of options: Tversky and Shafir (1992) found that individuals tend to defer making decisions when the set is too large or when the decision is difficult, which suggest the existence of a cost. Moreover, it can also be explained by the decrease in discriminability between options, which decreases the precision of the utility given to each option (DeShazo and Fermo, 2002).
Another external factor which can affect valuation is the impact of options’ attributes and the choice set used. Three main effects have been well described in economy: the attraction, the compromise and the similarity effects. All of those effects occur when two equally preferred options (choice is 50% for each) varying on two dimensions are presented with a third option. The effects are dependent on the quality of the third option. The attraction effect, discovered by Huber, Payne and Puto (Huber et al., 1982) corresponds to the situation were a third option added is similar to B but inferior to it on one of the two attributes, this addition results in a probability of choosing B higher than 50% (Figure 7, left). The compromise effect (Simonson, 1989) arises when the third option has the same global utility as A and B but more extreme than B. In this case, the percentage of choice is also biased toward B (Figure 7, middle). Finally, the similarity effect occurs when the third option is very similar to B: the preference is shifted toward A (Figure 7, right). The mechanisms underlying those effects are still debated but they are critical examples of how the context can influence the valuation process.
Models of decision-making
Values are needed to make a decision. Indeed, if I have to make a choice between an item A and an item B, I need to compute the value of A and the value of B, with each value being affected by factors seen in the last part; then, I will need to compare the two values in order to select the item which has the highest value. However, I often mentioned that economists developed their theories to explain inconsistencies in choices. Indeed, observed choices often violate the principle of choosing the highest subjective value (‘argmax’ rule: choosing the maximum). Thus, how we compare values and how a decision is made is a topic which has yielded a growing interest in the last decades and several kinds of models have been developed to propose psychological mechanisms underlying decision-making. I will focus on two types of models, widely used in the field. The first kind is what I will call ‘static models’ which proposes mechanisms to explain choices and the second kind concerns models which explain both decision and time of decision, I will refer to it as ‘dynamical models’. Those two types of models have their origins in psychophysics and they were originally focusing on perceptual decision-making but they have been recently extended to value-based decision-making.
Static models of decision-making Basic fitting
As we saw, choices cannot be explained and modelled by a simple deterministic ‘argmax’ rule since all factors such as context cannot be taken into account in the computation of the subjective value. Then, one solution is to use probabilistic methods to explain them by fitting a sigmoidal function such as the softmax function, already used in learning theories (Luce, 1959). These methods allow drawing psychometric functions which capture the fact that the number of inconstancies in choices increases when values are close (Snodgrass et al., 1985). I will use this kind of logistic fit in most of the experiment I will present to analyze choices, since it is one of the first manipulation check one should do when investigating choices. From this kind of fit, we can have access to two parameters: an intercept indicating if there is any bias in choices and a temperature usually called β, which is capturing the noise in choices7. However, this function is not bringing any mechanistic view on how choices are made, and that is why more mechanistic models have been developed.
Signal Detection Theory (SDT)
The initial purpose of the Signal Detection Theory was to explain sensory decisions with uncertainty (Peterson et al., 1954; Tanner Jr and Swets, 1954). It assumes that there are four kinds of possible answers when we have to detect the presence of a stimulus: hit (respond yes when the stimulus is present), correct rejection (respond no when the stimulus is absent), false alarm (respond yes when the stimulus is absent), and miss (respond yes when the stimulus is present). From the proportion of those kinds of responses, one can compute the sensitivity d’ (or discriminability) to the stimulus, which can be seen as a pure perceptual parameter, and a criterion β, sometimes called ‘response bias’, which can inform whether the subject is ‘conservative’ (tendency to say ‘no’: she wants to avoid false alarms, with the risk of missing some detection) or ‘liberal’ (tendency to say ‘yes’: she wants to avoid misses, with the risk of saying yes while the stimulus is absent) (see Figure 10).
Table of contents :
ACKNOWLEDGEMENTS
RÉSUMÉ
SUMMARY
TABLE OF CONTENTS
Chapter 1: Theoretical background
1. Subjective values & preferences
A. Brief history of subjective value investigation
B. Values in decision-making
C. Subjective value properties
D. Models of decision-making
E. Summary & open questions
2. Neural substrate of values
A. Neuroanatomy
B. Brain activity measurements
C. Value coding in the brain
D. Specific Properties of the Brain Valuation System
E. Summary & open questions
3. Using values in choices
A. The case of simple choices
B. Sequential sampling models & neural implementation
C. Debates around the value-based decision-making network(s)
D. Summary & open questions
Chapter 2: Experimental studies
1. Behavioral investigation of subjective value measurements
A. Introduction
B. Article
C. Discussion
2. Investigation of the Brain Valuation System Local Field Potentials .
A. Introduction
B. Article
C. Discussion
3. fMRI investigation of the Brain Valuation System during decisionmaking
A. Introduction
B. Article
C. Discussion
Chapter 3: General discussion
1. Methodological approach
A. On the advantages of using model fitting & model comparisons.
B. LFP & BOLD: Similarities & differences
2. Theoretical implications
A. On the behavior
B. Properties of the Brain Valuation System
C. Components of the Brain Valuation System
3. Acknowledged limits & open remarks
A. Focus on the Brain Valuation System
B. Focus on ‘positive’ values
C. What is value in the brain?
4. Conclusion
References
