Neurochemical Mechanisms of Reinforcement Research Paper

2. Characterizing The Brain Reinforcement Mechanism

The problem of characterizing the brain reinforcement mechanism has two main parts: the first is to identify the neuronal substrate that performs the reinforcing function (reinforcing substrate) and the second is to identify the neuronal substrate that is modified by the reinforcement process (target substrate). Since any goal object can reinforce any behavior in an animal’s repertoire, it seems likely that there is (a) convergence of goal-object input to the reinforcing substrate, and (b) divergence of reinforcing output from this substrate to the target substrate. Here we briefly summarize the substantial published literature that has been devoted to the identification of the reinforcing substrate. Until very recently, little consideration has been given to the question of the target substrate, but it may be useful to review some initial findings and to consider the directions that research on this problem may take.

2.1 Neurochemical Specialization

The idea that reinforcing functions are specialized neurochemically has guided research in this field for more than 30 years. The hypothesis that certain dopamine and opioid peptide brain cells may serve as reinforcing neurons is supported most directly by evidence from brain self-stimulation and drug self-administration experiments.

2.2 Electrical Self-Stimulation Of The Brain

In self-stimulation experiments, animals work to deliver electrical stimulation to their own brains through permanently indwelling electrodes. In the absence of other sources of reward, the reinforcement for self-stimulation behavior must arise from the neuronal activity that is excited by the electrical stimulus. If so, it would be logical to assume that some of the neurons under the electrode tip actually are the reinforcing neurons that mediate the effects of natural reinforcers or at least are neurons that directly excite them.

High self-stimulation rates are observed when electrodes are implanted in regions containing dopamine (or opioid peptide) cell bodies or pathways. In particular, self-stimulation tightly overlaps the distribution of dopamine cells in the ventral tegmentum and substantia nigra. Self-stimulation closely follows the anteriorly projecting dopamine fibers through the hypothalamus, but it correlates somewhat less closely with the dopamine terminal fields in the forebrain. The involvement of norepinephrine neurons in self-stimulation is more controversial. Although many laboratories report self-stimulation from sites in the vicinity of the locus coeruleus, it has not been possible to establish convincingly that the noradrenergic neurons that make up this nucleus are responsible for the reinforcing effect. Mapping of opioid peptide sites for self-stimulation is consistent with the idea that certain beta-endorphin and dynorphin neurons are involved in self-stimulation, but these studies are still in an early stage. The dopamine-opioid peptide reinforcement hypothesis also is supported by pharmacologic experiments. Antagonists of dopamine and opioid peptides, such as haloperidol and naloxone respectively, should block chemical transmission of reinforcement messages. In support of the model, there are many reports that these drugs selectively block self-stimulation.

Interestingly, the pharmacology of avoidance behavior (a prototypical example of negative reinforcement) closely overlaps that of self-stimulation (exemplifying positive reinforcement). In particular, avoidance and self-stimulation behaviors are selectively facilitated by dopamine agonists, such as amphetamine, and both are selectively inhibited by dopamine antagonists, such as chlorpromazine. Such evidence suggests that a common dopamine substrate may underlie both positive and negative reinforcement processes.

2.3 Intravenous Drug Self-Administration

In self-administration experiments, behavior is reinforced by systemic or central injections of neurotransmitters or drugs. Although thousands of chemical substances are available, animals and humans avidly self-administer only a few. These self-administered substances may properly be termed pharmacologic reinforcers and it is interesting to ask why these chemicals are selectively associated with behavioral reinforcement. It is no coincidence that most powerful pharmacologic reinforcers have the ability to mimic or release the hypothesized natural reinforcement transmitters. Thus, many dopamine and opioid receptor activators are known to support self-administration behavior. In particular, selective agonists acting at µ and δ opioid receptors function as positive reinforcers, as do the naturally occurring opioid peptides, βendorphin and dynorphin. Similarly, agonists acting at dopamine D₁-like and D₂-like receptors, such as apomorphine (D₁ /D₂), piribedil (D₂), and SKF 82958 (D₁) are self-administered. The patterns of self-administration for many of these receptor agonists (particularly, D₁ and µ agonists) resemble the patterns observed when highly addictive stimulant drugs (e.g., cocaine and amphetamine) or opiate drugs (e.g., morphine and heroin) are self-administered. Hence it is reasonable to suppose that addiction to stimulant or opiate drugs may depend largely on their ability to activate or to enhance the activity of natural dopamine or opioid peptide reinforcement systems respectively. Finally, dopamine and opioid peptide antagonists, such as chlorpromazine and naloxone, not only appropriately block the reinforcing effects of agents that activate dopamine or endorphin receptors, but also appropriately block the reinforcing effects of stimulant and opiate drugs. Serotonergic agents generally are not self-administered but serotonin agonists and antagonists have been shown to alter self-administration of psychostimulants such as cocaine.

The self-administration method also has been used in attempts to identify the sites of action of stimulants and opiates. In some studies, animals are trained to self-inject cocaine or morphine directly into the brain in an attempt to find preferred sites for reinforcing effects. In other studies, dopamine or opioid receptor antagonists are injected into different brain regions in attempts to block intravenous self-administration of drugs of abuse. Results from several laboratories suggest that important sites of action for both stimulant and opiate drugs are concentrated in the ‘extended amygdala’ including brain regions on an axis from the nucleus accumbens to the central amygdala. Reinforcing actions also are reported after direct injections of cocaine in the prefrontal cortex, morphine in the ventral tegmental area, and dynorphin in the hippocampus. Evidently, target sites for the reinforcing actions of drugs are widely distributed in the brain.

A closer analysis of brain transmitter involvement in reinforcement has been accomplished using in vivo microdialysis to monitor neurotransmitter levels in various brain areas during drug self-administration. Each cocaine self-injection caused a large increase in dopamine levels in the nucleus accumbens and these dopamine levels usually fell substantially to a ‘trigger’ concentration before the animal delivered the next self-injection. The addition of heroin to the cocaine solution caused synergistic increases in nucleus accumbens dopamine levels that are consistent with the higher abuse potential of such ‘speed ball’ combinations.

3. Methodological Considerations

Although these anatomical and pharmacological findings are generally consistent with the idea that reinforcing functions are performed by specialized systems of dopamine and opioid peptide neurons, some important problems remain. Administration of pure substances in the self-administration experiments strictly controls the chemical nature of the reinforcing injections but because the distribution of injected transmitters to active sites cannot exactly duplicate that of naturally released transmitters, the ensuing pattern of receptor activation could be artifactual or misleading. On the other hand, while electrical activation of transmitter pathways during self-stimulation presumably releases the reinforcing chemical messenger in a relatively natural distribution at appropriate postsynaptic sites, the electrical stimulus also must cause the simultaneous release of many irrelevant transmitters and neurohormones, including some that are still unknown. Thus, in self-stimulation, identification of which cells and which transmitters are relevant to the reinforcement process is largely a matter of inference. Solutions based on mapping studies demonstrate self-stimulation in anatomically coherent systems and those based on pharmacological studies implicate specific neurotransmitters. However, the anatomical mapping data from most self-stimulation experiments are insufficiently detailed. At of the pharmacological reports can be criticized because they often fail to distinguish specific drug effects on reinforcement from nonspecific effects on motor performance or attention. In addition, some self-stimulation findings seem to contradict the dopamine theory of reinforcement. For example, lesions of the nigrostriatal bundle do not permanently eliminate substantia nigra self-stimulation and, if only motorically simple responses are required, hypothalamic self-stimulation survives extirpation of all forebrain dopamine terminals.

4. The Dopamine-Opioid Peptide Theory Of Reinforcement

Despite such difficulties, the dopamine-opioid peptide theory of reinforcement currently enjoys a widespread acceptance. First, many investigators are impressed by the powerful and reliable reinforcing actions that highly localized electrical brain stimulation and specific chemicals can produce. Second, investigators are impressed by the consistent picture of dopamineopioid peptide involvement in reinforcement processes that has emerged from the self-stimulation and self-administration data. Other neurochemical systems, in addition to dopamine and opioid peptides, also are likely to participate in reinforcement functions. For example, brain cannabinoid and nicotine receptors are probably involved in the reinforcing actions of marijuana and nicotine respectively. As additional receptors are identified as candidates for reinforcing actions, questions arise regarding their primacy and specific roles in reinforcement. Evidence now exists, for example, that several transmitter systems (e.g., cannabinoid, nicotine) have receptors localized on dopaminergic neurons that in turn may represent the final common pathway for reward. Another indication of a final common pathway utilizing dopamine is that agents that directly stimulate dopamine systems (e.g., amphetamine or cocaine) are more robustly self-administered than are agents that indirectly stimulate dopamine neurons (e.g., marijuana or nicotine).

Although a final common dopamine reward pathway remains a possibility, the crucial importance of reinforcement for survival makes it more likely that redundant neurotransmitter systems are involved. Most research on reinforcement has focused on the familiar transmitter systems mentioned above but complete understanding of reinforcement neurochemistry must await the likely discovery of novel transmitters. For example, several hundred G-protein coupled receptors have been cloned but their endogenous transmitters have not yet been identified. Some of these ‘orphan’ receptors may well play a role in reinforcement and their endogenous ligands (when elucidated) will serve as novel reinforcement transmitters. Such discoveries will lead to a more subtle and comprehensive understanding of reinforcement.

5. Identification Of The Target Substrate

As noted, little theoretical or experimental effort has been devoted to the identification of the substrate that is modified by the reinforcement process. What are the neuronal targets of the reinforcing neurons? Because it is operant behavior that is reinforced, it is plausible to assume that behavioral substrates are major targets. It is commonly believed that operant conditioning involves the strengthening or reorganization of complex (but yet to be identified) neuronal circuits, which presumably form the substrates of reinforce able responses. According to this conventional view, the whole response is the unit of reinforcement. Where in the rat’s brain, for example, would one find the circuit for a lever-press response? How might this circuitry be reorganized when lever pressing is reinforced? These anatomical and physiological considerations, as well as difficulties at the behavioral level associated with response definition, have led some scientists to question the idea that elaborate neuronal circuits are the functional units for reinforcement. According to B. F. Skinner, a more useful conceptual scheme recognizes the ultimately continuous nature of operant behavior. In this scheme, behavior is construed–-not as a series of discrete acts–-but rather as a continuous flow of activity and interaction with the environment, which arises from the largely spontaneous activation of infinitesimal behavioral elements. It is these elements, or ‘behavioral atoms,’ and not whole responses, that are the units strengthened by reinforcement. If so, and if atoms of behavior can be represented by the activity of individual neurons, then it could be argued that it is the behavior of individual neurons, and not of neuronal circuits, that is directly modified by reinforcing signals.

The cellular reinforcement hypothesis is currently under test in our laboratories by use of a novel ‘in vitro reinforcement’ paradigm. This method employs training procedures closely analogous to those of behavioral operant conditioning to answer the question: Can the activity of relatively isolated brain cells in hippocampal slices be reinforced by local micropressure injections of dopamine or other reward related transmitters or drugs? It was found that the bursting responses of individual pyramidal neurons in the CA I layer were in fact progressively increased by burst-contingent applications of dopaminergic and cannabinoid receptor agonists. The bursting responses of CA3 units were similarly increased by opioid receptor agonists. The same microinjections, administered independently of cellular activity, failed to facilitate and frequently suppressed cellular bursting. This observation, and the fact that glutamate (an excitatory transmitter that is not associated with the behaviorally-reinforcing effects of drugs) failed to increase bursting, indicated that general stimulation of cellular activity is an unlikely explanation of the facilitatory action of the burst-contingent injections. Reinforcement delays exceeding 200 ms largely eliminated the reinforcing action of dopaminergic agonists. These findings support the conclusion that the behaviorally-reinforcing effects of dopaminergic, opioid, and cannabinoid drugs can be modeled in vitro at the cellular level.

Bibliography:

1. Gardner E L 1997 Brain Reward Mechanisms. In: Lowinson J H, Ruiz P, Millman R B, Langrod J G (eds.) Substance Abuse: A Comprehensive Textbook. Williams & Wilkins, Baltimore, MD, pp. 51–85
2. Klopf A H 1982 The Hedonistic Neuron: A Theory of Memory, Learning, and Intelligence. Hemisphere Press, New York
3. Self D W, Stein L 1992 Receptor subtypes in opioid and stimulant reward. Pharmacology Toxicology 70: 87–94
4. Skinner B F 1953 Science and Human Behavior. The Free Press, New York
5. Stein L 1978 Reward transmitters: Catecholamines and opioid peptides. In: Lipton M A, DiMascio A, Killam K I (eds.) Psychopharmacology: A Generation of Progress. Raven Press, New York, pp. 569–81
6. Stein L, Xue B C, Belluzzi J D 1994 In vitro reinforcement of hippocampal bursting: A search for Skinner’s atoms of behavior. Journal of the Experimental Analysis of Behavior 61: 155–68
7. Wise R A 1996 Addictive drugs and brain stimulation reward. Annual Review of Neuroscience 19: 319–40