The Guess Worker

1) 3.7. 2011

Dear Professor Berridge,

I am interested in your ideas about "wanting" and "liking". I have a slightly different perspective which I would like to communicate to you.

I agree with you that liking and wanting should be distinguished, that they are important states of mind before receiving a reward and that wanting in particular has a pivotal role in reward seeking. I also agree that dopamine is in some way connected to wanting.

However, where I differ from you is in my understanding of the nature of wanting itself. I believe that physiologically and biochemically wants are identical to pains. This may seem counterintuitive, but we should remember that wants can actually feel painful if they strengthen into desires and longings. Usually, though, the pain of a want is experienced less intensely and more fleetingly than most physical pains. This means that we often don't recognise wants as being painful.

Pain is the only directly motivational instrument of consciousness. Pain makes animals take actions until there is no more pain. In contrast, pleasure (the other instrument of consciousness) cannot motivate directly, because there is no reason why an animal which already is receiving pleasure would be motivated to get more pleasure.

Seeing as motivation requires pain and seeing as there is a connection between motivation and dopamine levels, this suggests there is a link between pain and dopamine. In my opinion falling concentrations of dopamine in certain synapses could be the signal which is interpreted by the consciousness as pain.

I am aware that you consider any connection between dopamine and pleasure unlikely. But the jury is still out on this. In my view, facial expressions upon giving sweet or sour solutions does not indicate liking or disliking. If these responses are caused by innate associations in the brain, they will be unconscious and therefore can't involve pain or pleasure.

Whereas falling concentrations of dopamine signals pain, rising concentrations might signal pleasure. In the case of an animal being motivated to take a food reward, the sequence of events might be as follows:

Because of associations previously formed between the appearance of food and pleasure, the visual stimulus of food results in an increase in dopamine concentration in the synapses. This increase is experienced as liking (or attraction) to the food. However this sensation, being pleasurable, is not motivational. The rise in dopamine level must be followed by a fall (due to uptake) and it is this fall which provides the motivation. The fall is interpreted as pain, which here is experienced as a want.

If we think about the motivation to take a reward, we ought to realise that it is not the reward that motivates, but rather the absence of the reward. If the reward represents pleasure, then it is reasonable to suppose that the absence of the reward means pain. Pain and pleasure therefore are flip sides of the same coin. But it is only pain which motivates an animal to strive for a reward.

I hope these ideas are in some way useful and I would welcome any feedback from you.

Regards, Philip Ludikar

Prague, Czech Republic

2) 5.7.2011

Dear Dr. Ludikar,*

Thank you for your thoughtful message.

Your idea that low dopamine creates a painful or unpleasant state is quite intuitive. It fits well within a longstanding tradition in psychology of viewing motivation as accomplished via unpleasant drives, and that we act to achieve a drive reduction to lessen the pain or unpleasantness.

Yet there are powerful reasons, in my opinion, to think that that drive reduction is not the way the brain actually organizes motivational states in most cases. There are also reasons to think that motivation is actually most powerful when dopamine is elevated, rather than when dopamine is low.

I'll attach a paper that discusses drive reduction and its problems. Also another recent study that indicates dopamine elevation (produced by a microinjection of amphetamine in the nucleus accumbens) creates elevated 'wanting' in both the brain and in motivated behavior. It's important to note that the higher 'wanting' occurs when synaptic dopamine would still be elevated while amphetamine, which reverses the re-uptake transporter mechanism to release more dopamine onto post-synaptic neurons, is still active and before the post-drug depression begins.

I hope this is helpful.

Good luck in your projects, and best wishes,

Kent Berridge

3) 5.7.2011

Dear Professor Berridge,

Thank you indeed for your reply. I will read the enclosed papers with interest.

I would just clarify that in my view the absolute levels of dopamine are not important. Rather, rising dopamine is interpreted as pleasure and falling dopamine as pain. After stimulation of neurones, the up-phase of a dopamine spike would correspond to pleasure (ie. liking) and the down-phase would correspond to pain (ie. wanting). I believe the baseline level of dopamine plays a key role in the regulatory mechanism. This operates through the tendency of dopamine to rise or fall to the baseline.

Best wishes, Philip Ludikar

4) 11.7.2011

Dear Professor Berridge,

I have read both papers you sent me and I would like to suggest that the results of your recent work do not invalidate the model I proposed. In fact, I believe they corroborate my ideas.

First, I would point out that in broad terms I agree with your interpretation. It is likely that the firing response to CS+2 corresponds to what you call "incentive salience". However, unlike you, I believe that incentive salience consists of two consecutive components, the first of which we could call "attraction" and the second "wanting". These components could be identified if we were to follow events in the synapases after firing. Here the initial rise in dopamine would be followed by a fall and the up-phase of this spike would be interpreted as pleasurable attraction and the down-phase as a "painful" want.

In your email you wrote that "motivation is actually most powerful when dopamine is elevated, rather than when dopamine is low". I agree and this is entirely consistent with my model. We can expect elevated levels of dopamine to generate sharper falls. These falls will occur simply because of equilibrium pressures. If dopamine is already elevated, then further release of dopamine into synapses will be counteracted by (let's say) diffusion, even if transporters are blocked. This means that the rate of dopamine decline after the rise is greater than otherwise. The gradients of rises and falls determine the intensity of pain and pleasure, and so wanting will be felt more intensely in conditions of elevated dopamine.

The same pattern of synaptic events occurs after intraoral application of sucrose. Elevated dopamine levels cause a dampening of the rate of dopamine rise after firing. "Liking" therefore will be experienced less intensely than at normal levels of dopamine. In other words, as your results show, we can expect wanting to be increased and liking decreased after injections of dopamine-stimulating drugs.

Post-drug depression obviously has nothing to do with these events, but can be explained similarly. Removal of amphetamine results in a decline in dopamine. We can therefore expect the process to be an unpleasant experience.

With regard to your procedures, I would make one comment about the reliability of orofacial expressions as an indication of liking and disliking. Experience tells us that such expressions occur concurrently with feelings of liking and disliking. This is undoubtedly true in normal circumstances and probably is true in all trial conditions in your recent paper. However, in one state at least there can be no concurrency. This is the state of dopamine depletion. Orofacial expressions are likely to be reflex actions caused by innate associations. This means that stimuli will produce such reflexes even in dopamine-depleted animals. With normal levels of dopamine the animals would also experience liking or disliking. But if they are dopamine-depleted, these feelings cannot occur.

In conclusion - the model I have proposed appears to be consistent with your experimental results. The model is not a drive reduction hypothesis in the sense you describe in your 2004 paper. In contrast to such hypotheses and in agreement with your ideas, liking as well as wanting are essential components of reward seeking motivation. However, in my view there is a fundamental difficulty with your concept of incentive salience. It is hard to see how, by itself, the tagging of stimulus representations in the brain could provide motivation. If smells from a bakery trigger thoughts of lunch, how would those thoughts then motivate me to go and have lunch? I believe the missing element in incentive salience is my conception of a want - that is, a kind of pain. Such a want would be able to provide motivation.

As before I would greatly welcome any feedback from you, including any evidence or arguments which you feel contradicts the above hypothesis.

Best wishes, Philip Ludikar

5) 11.7.2011

Dear Dr. Ludikar,

I will think about your ideas of transient drops.

However, in the case of 6-OHDA depletion of dopamine, I think the orofacial reactions remain sensitive to hedonic evaluation.

For example, new taste aversion learning and hedonic elevation by benzodiazepine is still possible as found here, and the learning shift in palatability in particular requires forebrain participation.

Kent Berridge

6) 22.7.2011

Dear Professor Berridge,

After reading your 1998 paper I must revise my position: your results very probably do show that in dopamine-depleted rats orofacial reactions remain sensitive to hedonic evaluation. This clearly presents a serious challenge to my hypothesis. However, before modifying or abandoning my ideas I would suggest that, given certain events in the synapses, the model could still work.

Even though dopamine-depleted rats have very low concentrations of dopamine in their brains, dopamine levels could still decline in the synapses. V. Denenberg et al (in "The role of DA in learning, memory and performance of a water escape task", Behavioural Brain Research 2004) report that L-DOPA injections into the brains of dopamine-depleted rats (with around 1% of wild type concentrations) "bring DA content up to 9.1%, which is sufficient to generate hyperactivity of food and water consumption."

In other words, even relatively low concentrations are enough to restore motivation. What is important, in my view, is not the absolute level of dopamine but whether there is sufficient dopamine to allow for rises and falls. I believe that even in severely dopamine-deficient states declines in dopamine could be enough to facilitate the learning of aversive reactions.

Conscious learning (ie. learning via pleasurable or painful evaluations), in my opinion, is a process of transforming conscious responses to stimuli into unconscious responses. In this process associations become fixed or "stamped in", so that in future animals can respond quickly (and unconsciously) to learned stimuli.

Because I believe consciousness to be the brain's faculty of motivation, I regard learning (which possibly is always to some extent conscious) to be a motivated process. Somehow "pain" motivates animals to make and fix associations between sets of information. So according to my hypothesis, for learning to occur, there have to be falls in dopamine levels in synapses.

In dopamine-depleted rats the extremely low concentrations of dopamine may be a physical barrier to learning responses to hedonic rewards. This is because the quantities of dopamine may be too inadequate to deliver the necessary rise (experienced as "attraction") and fall (experienced as "wanting").

In contrast to hedonic rewards, aversive stimuli would only depress levels of dopamine. If the stimuli are strongly aversive they may be enough to depress even very low concentrations of dopamine.

If this idea is correct, it may explain the report by M.Frank et al ("By carrot or by stick: cognitive reinforcement learning" Science vol.306, 2004) that patients with Parkinson's disease who are off medication (ie. they have low dopamine levels) are better at learning from negative outcomes than than they are from positive outcomes. Conversely patients on medication (ie with higher dopamine levels) are more sensitive to positive outcomes.

Falling dopamine levels, then, may be the reason why affective reactions in dopamine-depleted rats can be modulated by aversive conditioning. However, why diazepam enhances hedonic reactions in such rats cannot be explained so easily. With increased firing of neurones, perhaps temporary increases in dopamine concentrations can be generated in the synapses. Without knowing exactly what is going on here, it is difficult to hypothesise.

So as not to make this email too long, I'm not going to comment now on other points you made in your 1998 paper. But, as always, I would welcome any feedback from you.

Kind regards, Philip Ludikar

7) 22.7.2011

Dear Dr. Ludikar,

Yes, I agree that low levels of dopamine around 10% are probably enough to support motivation function. Similarly I have heard that Parkinson’s patients do not begin to show motor symptoms until their DA declines to below 10%.

The 6-OHDA rats were of course depleted further, sometimes down to levels of 1%. The most depleted individuals were still similar to normal rats in taste reactivity, learned aversions, and benzodiazepine enhancement (which probably acts primarily in the brainstem to increase taste palatability, and so might not be quite as strong as enhancement evidence as could be wished).

I’m sure you also know of the series of other studies by Richard Palmiter’s lab on his dopamine-deficient mutant mice. My favorite is their 2005 paper in Behav Neurosci showing that the mice learn quite normally in a maze without dopamine as long as they are on caffeine. In any case, they have shown that several types of reward learning seem to occur relatively normally even in the virtual absence of dopamine (again probably 1 or 2% at most, possibly less).

Still, I agree that phasic dopamine manipulations will be interesting to study. The new optogenetic techniques that use laser light to activate specific neurons might be promising for that. We are now beginning to try the laser technique to manipulate target neurons in the nucleus accumbens and striatum (either D1 direct neurons or D2 indirect neurons selectively). The tyrosine hydroxylase Cre-marked mutants needed to manipulate dopamine-release are not yet so readily available. But I’m sure that situation will improve in the next year or so, and then results relative to your idea might be forthcoming.

Best wishes,

Kent Berridge

8) 28.7.2011

Dear Professor Berridge,

I wasn't able to get hold of the 2005 paper you mention but I did read your commentary in the same issue. So I believe I have the most essential information.

What could be significant here is that whereas responses to aversive stimuli can be learnt by dopamine-depleted rats without the application of any other drug (as in your 1998 paper), in this experiment the response to the hedonic stimulus is learnt in the presence of caffeine. I would explain this in the same way as before: even in dopamine-depleted rats aversive stimuli can cause small declines in dopamine and these declines could enough to facilitate learning. Rises in dopamine levels are however much less likely - which means that these rats normally can't learn responses to hedonic stimuli. However caffeine may be able to generate localised increases in dopamine in some synapses even in severely depleted states. Therefore, in this indirect way, caffeine may be able to promote the learning of responses to hedonic stimuli.

So the key question is: can responses to hedonic stimuli be learnt in dopamine-depleted rats without the application of other drugs? If they can't, this could indicate that the model I proposed is correct.

Such an experiment could be difficult to design simply because of the extreme lack of motivation of dopamine-depleted animals. But I would contend that it is the animals' lack of motivation which actually makes them unable to learn in the first place. In my view, as I have mentioned, learning is a motivated process. My reasons for believing this are merely anecdotal - but I think you would agree with me that problems which can't be solved easily tend to generate unpleasant feelings in us and, conversely, the solving of a difficult problem can produce joy or euphoria (ie. the eureka effect).

In any case, I look forward to following your future projects and I wish you the best of luck in them.

Kind regards, Philip Ludikar

9) 28.7.2011

Dear Dr. Ludikar,

I quite agree with your final paragraph, that learning is motivated and that removal of motivation will disrupt much ordinary learning. Also, that it is very difficult to test for reward learning in absence of dopamine or other drugs.

However, one step toward that test was provided by an earlier study by the Palmiter lab:
http://www.jneurosci.org/content/23/34/10827.full
which you perhaps will also find interesting.

Good luck in your projects and best wishes,

Kent

10) 1.8.2011

Dear Professor Berridge,

I agree with you that the method used by Richard Palmiter's lab might provide a way to test for reward learning in the absence of dopamine. It must be said, though, that even here the mice are so hypoactive that it may be difficult to obtain convincing results.

I would be interested to know what you make of the results of this particular study. As I interpret your ideas (please correct me if I'm wrong) these results challenge your hypothesis as much as they do mine. We both agree that for hedonic rewards, wanting is necessary for motivation and that wanting is dopamine-dependent. If, as claimed, amphetamine is really able to eliminate all dopamine, then according to both our models, the mice should not be able to experience wanting at all. Therefore they could not be motivated to do anything. But in reality these mice are still motivated enough to choose sucrose in preference over water.

Of course it is possible that motivation is also supported by another system (or perhaps "systems") which is independent of dopamine. The only reservation I would have about this idea is that biological systems are unlikely to be more complex than necessary.

By the way, if you wish, feel free to address me by my first name.

Best wishes, Philip

11) 2.8.2011

Dear Philip,

I think the message from the Cannon paper is that the mutants have less than 1/10th normal motivation. They hardly drink anything at all in undrugged state. Not nearly enough to sustain life.

But of the small amount they do drink, they prefer the sucrose to water, suggesting they have registered the hedonic difference.

It is of course an interesting issue why they drink at all.

Best wishes,

Kent

12) 2.8.2011

Dear Kent,

Please forgive my persistence, but I would argue that it is important to know why the mice remain motivated despite the apparent absence of dopamine.

Even though I agree with you that liking and wanting are distinct components, in my view there must be a connection between them. Somehow greater liking must provoke greater wanting. Because the mice like sucrose more than water, they are more motivated to return to the sucrose spout than they are to the water spout. I believe that to understand wanting and liking fully we must understand the nature of this connection between these two components.

I would also add that if there are indeed two motivational systems, one dopamine dependent and the other dopamine independent, then wanting and liking could still arise in the dopamine system in the way I suggested. Of course, in the non-dopamine system wanting and liking would have to be caused in a different way, but even here, as the Cannon paper shows, the connection between wanting and liking is apparently preserved.

Kind regards, Philip

13) 2.8.2011

Dear Philip,

Yes, that's quite reasonable.

Kent

* I don't have a PhD. However, I didn't consider it necessary to correct Professor Berridge's misapprehension.