Psychology and Ethology of Learning Research Paper

There are two major traditions in the study of learning in animals, one within experimental psychology, and one within zoology, especially ethology. Although for several decades the orientations and work of these two traditions seemed antithetical, more recently the study of animal learning has profited from a greater synthesis of these approaches. In this research paper we first develop a few key ideas within each of the separate traditions and then provide three case studies that show how a more synthetic approach, combined with an interest in the neural organization of learning, can provide important insights into the nature of learning. Because this is a handbook of psychology, we emphasize the tradition of experimental psychology.

Ethological Approaches To Learning

Ethology has made a key contribution to the study of animal learning. At the same time, the investigation of learning has played an important role in the development of ethological theory. The many exchanges that occurred between ethologists and experimental psychologists just after World War II were especially important in sharpening the research approaches that were adopted by ethologists for the study of learning (e.g., Lehrman, 1953; Schneirla, 1956). In this section we consider generally the ethological approach to the study of behavior and more specifically how this approach manifested itself in studies of animal learning conducted by ethologists. We address these issues by first asking, “What is ethology?” and then discussing the ethological view of learning and its application to the study of learning.

What Is Ethology?

Perhaps the briefest definition of ethology to be commonly adopted is that it is “the biological study of animal behavior” (e.g., Immelman & Beer, 1989; Tinbergen, 1963). This definition fails to capture important nuances that distinguish ethology from other fields in the behavioral and neural sciences that also consider nonhuman animals as subjects. The qualifier that ethology represents a zoological approach to the study of behavior provides a useful additional insight (Beer, 1963; Thorpe, 1979). Zoologists are intrinsically interested in the study of animals for their own sake and on their own terms, rather than as a model system to understand some basic biological process that will be broadly applicable to many life forms including humans. In other words, viewing behavior in zoological terms means that ethologists take an animal-centered view of the study of behavior.

Ethologists therefore first and foremost are interested in understanding how an animal behaves under natural conditions. Thorpe (1979) has observed that ethology along with ecology can be viewed as the scientific legacy of a fascination with natural history in Western culture that can be traced back at least to the 12th century and Francis of Assisi. It is this natural-history orientation toward behavior that all authorities agree is one of the central features of ethology (e.g., Hinde, 1982).

How is a natural-history orientation manifested in the scientific study of behavior? Combining a thorough description of behavior under natural conditions with an understanding of the sensory abilities (and limitations) that an animal possesses is an essential first step in any detailed study of behavior. Such descriptive data are usually collected under field conditions where one can appreciate the challenges that an animal faces. One’s appreciation for the possible behavioral capabilities an animal possesses increases markedly when one actually experiences the challenges that an animal normally faces. Adopting a natural-history orientation does not mean that one eschews experimentation. On the contrary, ethologists have often performed experiments and manipulations of various sorts. But an ethologist will always be concerned about the relevance of such manipulations to the natural situation. Just because one can reliably observe an animal engage in certain behaviors under certain environmental conditions does not mean that one has learned anything of value of the causes of behavior. A good ethologist is always concerned about artifactual responses that might be observed under artificial conditions that can fool one about the actual capabilities that an animal possesses. Finally, the natural-history orientation adopted by ethologists means that they are interested in not only the causes and development of behavior (sometimes called proximate causes), as are many other behavioral scientists, but also the adaptive significance and evolution of behavior (sometimes called ultimate causes of behavior).

Many ethologists have stressed that this interest in multiple levels of analysis is a key aspect of core ethology (e.g., Hinde 1982; Lorenz, 1981; Tinbergen, 1963). Lorenz (1981) phrased this notion in a way that clearly illustrates the link between ethology and Darwin. He contended that ethology applies to behavior “all those questions asked and those methodologies used as a matter of course in all the other branches of biology since Charles Darwin’s time” (p. 1). This interest in the evolution of behavior has framed the ethological approach to the study of learning and has also set the stage for some of the conflicts that occurred between ethologists and experimental psychologists.

To understand ethological approaches to the study of learning we should be familiar with some basic ethological concepts about how behavior is organized. One important notion is that behavior is often packaged into highly stereotyped patterns known as fixed action patterns (Lorenz, 1950; Tinbergen, 1951). As observed by Lorenz (1950) and others, the patterning of these fixed action patterns is generally species-specific and can therefore be used as a trait along with morphological and genetic characters to build a taxonomy. Fixed action patterns are often preceded by more variable behavioral responses, known as appetitive responses, that put the animal in the situation to express a fixed action pattern (Craig, 1918; Hinde, 1970; Marler & Hamilton, 1966). The fundamental idea is that some stereotyped behaviors result in a functional outcome that is associated with a reduction in motivation whereas other more variable behaviors allow an individual to converge on this functional outcome (Timberlake & Silva, 1995). Although dichotomizing behavior in this way is problematic in some cases (Hinde, 1953), the distinction has been useful to both ethologists and experimental psychologists for the elucidation of the mechanisms mediating many motivated behaviors such as food-seeking and ingestive behavior (Timberlake & Silva, 1995). Furthermore, the study of these species-typical motor behaviors has provided us with insight into how motor systems are organized by the central nervous system to implement complex patterns of behavior (Hinde, 1953; Tinbergen, 1951).

An idea closely related to the fixed action pattern, which was also articulated early in the history of ethology, is that such species-typical behaviors are elicited by environmental stimuli in a highly selective manner. A famous example concerns the territorial responses of male robins. Lack (1939) presented male robins on their territories with a stuffed juvenile robin that had drab brown plumage, with a stuffed male robin with a red breast, or with just a bunch of red feathers. The territorial male robin responded aggressively to the model of the breeding male and ignored the model of a juvenile robin. However, his response to the bunch of red feathers was nearly equal to his response to the male model. This led Lack to conclude that the red breast was the key stimulus out of the myriad of stimuli that might be relevant that the male robin used to guide his aggressive responses. Stimuli like these are known as sign stimuli. Many other examples of these sorts of highly selective responses to stimuli in the environment have been reported since they were first described in detail by von Uexkull, a teacher of Lorenz, in the 1920s and 1930s (see Schiller, 1957). A sign stimulus is currently defined as “a single simple feature or a compound of a few simple features that provide only a small fraction of the total sensory input from a situation to an animal, but to which the animal’s specific reaction pattern is tuned, so that the stimulus selectively elicits this pattern” (Immelman & Beer, 1989, p. 270). The mechanistic basis of this selective stimulation has been studied to some extent. The simplest examples involve limitations in the relevant sensory receptors or in the tuning of sensory fibers so that the animal can detect only a very restricted part of the sensory world (Marler, 1961). In the case of the male robin, it is clear that the male is able to perceive colors besides red. One can therefore reject the obvious explanation that there is some sort of limitation of sensory receptors responsible for such selective responding. The neural basis of selective responding to stimuli can involve many different mechanisms besides just biases in sensory receptors, including learning processes such as sensitization or habituation.

What is apparent is that complex interpretations of sensory information are being made by the central nervous system to mediate many of these selective responses. Originally, it was thought that these sign stimuli worked via an “innate releasing mechanism” to release action-specific energy that would “fuel” behavioral production (e.g., Lorenz, 1950). This energy model of motivation has been criticized and is no longer held as valid by most ethologists (e.g., Hinde, 1970). However, there are certainly endogenous processes involved in the motivation of fixed action patterns. Most neuroethologists now avoid the terms motivation and drive and instead try to explain these endogenous processes in physiological terms.

The Ethological View of Learning

Many behavioral and neural scientists continue to think about variation among animals in hierarchical terms. Although the problems with this sort of reasoning have been discussed for many years, going back to Lovejoy’s classic monograph (1936; see also Hodos & Campbell, 1969), it is still not unusual to hear about different species being compared on the basis of being “higher” or “lower” vertebrates. Higher and lower in this context refers to the scale of nature in which mammals are high on the scale (with primates at the top) and birds reptiles, amphibians, and fish are lower on the scale (Hodos & Campbell, 1969). Invertebrates are of course lower still. There is also an implicit assumption associated with the embrace of hierarchical thinking that the ability to learn a particular behavior is somehow superior or more sophisticated than engaging in a similar behavior when it is unlearned to a large degree. Learning is thought to be associated with more complex nervous systems (such as those possessed by humans), so studying learning in any form will be valuable in understanding human behavior, and one might expect “higher” vertebrates to exhibit more learning and more complex learning than “lower” vertebrates.

With this reasoning in mind it is understandable how many experimental psychologists started to focus on the study of learning in a few convenient species of higher vertebrates so that generalizable principles of learning could be discerned. Even neurobiologists who adopted a reductionist approach to the study of learning and focused on invertebrate species such as the mollusks Aplysia (Hawkins & Kandel, 1984; Kandel, 1976) and Hermissenda (Alkon, 1983) argued that by studying the cellular and molecular bases of learning in these species, one could gain insight into fundamental processes of brain plasticity that would be widely applicable to many species, including humans. Again in this literature there is an implicit and in some cases explicit assumption that neuroplasticity is an advantageous trait and that the amount of plasticity that a nervous system is capable of is some gauge of the level of sophistication or complexity of that nervous system. At times it seems that in the scientific community the idea that learning and the associated neural plasticity must be a good trait is accepted as being as obvious as the notion that motherhood is a valuable trait as perceived by the community at large.

Ethology adopted a very different view of learning. If one views these issues from the perspective that the function of behavior is to maximize individual reproductive success, then the widespread occurrence of learning is potentially very dangerous. Animals in a given population, in a given habitat, have evolved a particular repertoire of morphological adaptations that make successful reproduction possible. Similarly, as previously discussed, ethologists have argued that species-typical behaviors are also adaptations that have evolved to complement these morphological adaptations to promote individual reproductive success. Learning is a way to bring about behavioral change based on experience. Behavioral change can potentially disrupt adaptive complexes of behavior and have disastrous consequences for the functional outcomes of behavior (i.e., reproductive success). Learning may indeed be advantageous or necessary for certain aspects of the behavior, but it should be highly controlled and limited so that the right sort of learning occurs at the right stage in the life history of the animal. It seems unlikely that open programs of neuroplasticity that facilitate unguided learning would be advantageous in many cases, and therefore they are unlikely to evolve very often.

The ethological argument concerning learning was perhaps most forcefully articulated by Konrad Lorenz, who pointed out that the “more complicated an adapted process, the less chance there is that a random change will improve its adaptiveness” (1965, p. 12). He goes on to point out that there are “no life processes more complicated than those which take place in the central nervous system and control behavior. Random change must, with an overpowering probability, result in their disintegration” (p. 12). These statements succinctly summarize the notion that learning should not necessarily be viewed as a useful trait. The related idea is that when learning does occur, it should be directed. With a rather high degree of invective, Lorenz states, “To anyone tolerably versed in biological thought, it is a matter of course that learning, like any function of comparably high differentiation and survival value, must necessarily be performed by a very species-specific mechanism built into the organic system in the course of its evolution” (p. 12).

The Ethological Approach to the Study of Learning: The Case of Imprinting

The ethological view of learning was perhaps best illustrated by the study of imprinting, first by Konrad Lorenz and then by a variety of other investigators (Bateson, 1966; Hess, 1973). Imprinting involves the formation of an attachment by progeny early in life for their mother and then later in life for a mating partner. Imprinting on a mother figure is known as filial imprinting, whereas an attachment for a mating partner is known as sexual imprinting. Imprinting has been studied in the most detail among bird species with precocial young, such as members of the galliform order (e.g., chickens, turkeys, or quail) as well as members of the anseriform order (e.g., ducks and geese). Imprinting clearly can be considered an example of a learning process because the object that a young animal becomes attached to and will follow around is based on the objects it experiences just after hatching or birth. Lorenz was famous for illustrating how he was able to get young goslings to form attachments to him. Many textbooks of psychology and biology include a picture of Lorenz leading a group of young goslings. This behavior resulted from the fact that the mother was removed so that at hatching the first moving object the goslings encountered was Lorenz, and they did indeed form an attachment with him. Similarly, he demonstrated how these goslings would later court him when they reached sexual maturity.

Filial imprinting can be measured in a variety of ways. The first way involves following the object of attachment. It can also be assessed by behaviors exhibited in the presence of the object (usually indicative of contentment) and behaviors exhibited toward other salient objects in the environment that it is not attached to (usually avoidance behaviors or even fear and panic). Sexual imprinting is measured later in life as a behavioral preference for a mating partner that resembles the object of filial imprinting to some degree.

When Lorenz investigated imprinting in the 1930s, he stressed the aspects of imprinting that made it different from general learning processes. He observed that the learning occurred with a minimal amount of experience (a single exposure for a limited amount of time is sufficient), that the ability to learn was optimal during a restricted period of time early in life (the so-called critical period), that this learning was irreversible (a new stimulus could not replace the original imprinting stimulus), and that it has effects on certain behaviors (sexual behaviors) that are not—indeed cannot be—produced at the time the learning occurs. However, modern results from a series of elegant experiments, carried out primarily by ethologists but also by experimental psychologists, indicate that the differences between imprinting and other examples of learning about single events may not be qualitative, but rather a matter of degree. Variables that influence the imprinting process also influence learning about single events in general. These variables include the quantity and quality of the stimulation, the duration of the stimulation, the animal’s state (age and past experience), and events that occur between when the animal has an experience and when it is tested (see Bolhuis, 1991, and Shettleworth, 1998, for reviews). The imprinting saga illustrates how the naturalhistory approach advocated by ethologists can lead to a rigorous experimental analysis of the variables influencing a learned behavior.

General Process Approaches to Learning

In contrast to the ethologists of the time, early experimental psychologists celebrated the role of individual adaptation to a changing world. Although they seldom articulated these attitudes, it is probably fair to say that for them, evolution provided only the raw materials, the bits and pieces of behavior, and that experience provided the opportunity for organized, adaptive behavior. Far from being a potential threat to survival, learning was the key to behavioral adaptation.

Indeed, as late as the middle of the twentieth century, many psychologists were optimistic that a full understanding of behavior, mind, and brain could be derived from a few basic and universal principles of learning. Thus, this orientation to the study of learning was sometimes termed general process theory. For example, Clark Hull wrote his classic Principles of Behavior (1943) “on the assumption that all behavior, individual and social, moral and immoral, normal and psychopathic, is generated from the same primary laws; that the differences in the objective behavioral manifestations are due to the differing conditions under which habits are set up and function” (p. v). Moreover, these “primary laws” were derivable from study of extremely simplified “model systems,” such as rats pressing levers and dogs salivating in anticipation of food. Early study of learning focused on animals not because of any intrinsic interest in animal behavior per se but because animal models provided a much greater degree of experimental control over past and present experience. Thus, in contrast to the animal-centered view of ethologists, experimental psychologists largely ignored their subjects’ natural histories and may be said to have adopted an experimenter-centered approach to the study of behavior.

The Reflex Tradition

These optimistic views were based in part on the successes of nineteenth-century physiological reflex theory (e.g., Sechenov, 1863/1965; Sherrington, 1906). In the extreme, the belief was that the activity of the brain (or mind) could be reduced to the translation of stimulus input into particular behavioral responses. Thus, the primary goal of psychology was to specify the relation between explicit stimulus inputs and response outputs. In this section we first consider the traditional models for the study of these input-output relations and then consider in depth some key ideas that have guided recent study of simple learning processes in animals within this tradition.

Classical and Operant Conditioning

The study of learning in experimental psychology has been dominated by two models, that of classical (or Pavlovian) and operant (or instrumental) conditioning. In classical conditioning a relation or contingency is arranged between two events over which the subject has no control. For example, in Pavlov’s laboratory (Pavlov, 1927), the sound of a metronome, the conditioned stimulus (CS), was repeatedly followed by the delivery of a food, the unconditioned stimulus (US), to a hungry dog. Eventually, the sound of the metronome alone came to elicit components of behavior previously controlled by the food (e.g., secretions of the stomach and salivary glands). In operant conditioning a relation is arranged between the animal’s behavior and the occurrence of some event (e.g., food is delivered to a hungry rat each time it presses a lever). In both cases, the arrangement of the appropriate contingencies results in the development of a conditioned reflex, habit, or association such that some stimulus comes to provoke a particular behavioral response automatically. Furthermore, the products of learning were characterizable in a single dimension, the strength of that habit, reflex, or association.

Although proponents of each model often attempted to describe the other model as a special case of their own (e.g.,C.L. Hull, 1943; Sheffield, 1965), some key differences are worth noting.Within the Pavlovian model, classical conditioning involved a process whereby the control of existing behavior is transferred from one stimulus to another. In the example described earlier, the metronome may be said to come to substitute for the food in controlling behavior (e.g., Mackintosh, 1974). Although it was widely recognized (C. L. Hull, 1943) that the conditioned (learned) response (CR) to the CS and the original, unconditioned response (UR) to the US need not be identical, the nature of learned behavior was nevertheless determined by the choice of US.

By contrast, within the operant model the learned response was assumed to be unconstrained by the reinforcer, limited only by the subject’s behavioral repertoire and the experimenter’s skill in extracting the desired behavior from that repertoire. The important feature of events that served as USs or reinforcers was not that they themselves unconditionally controlled behavior, but rather that they “stamped in” associations between stimuli (e.g., the sight of the lever) and responses (pressing the lever) on which they were made contingent, according to a law of effect (Thorndike, 1898). By this law, stimuli and responses are associated when they are followed by a “pleasurable event” (p. 103). More generally, behavior is governed by its consequences—its frequency depending on whether it has in the past produced reinforcing events. From the perspective of early learning theorists, it is this ability of animals’ behavior to be influenced by its consequences that formed the basis of adaptive behavior in individuals.

Both models emphasized experimental control over the animal’s experience and behavior. By isolating the animal from its natural environment in laboratories and still further in relatively small and sterile experimental chambers, influences on behavior other than those of immediate interest to the experiment at hand were thought to be minimized. These extraneous influences included not only distractions such as sights, sounds, or the presence of conspecifics, but also the opportunity to engage in other species-typical behaviors.

Learning, Motivation, and Emotion

Early study of conditioning was intertwined with the study of motivation. It was apparent that the effect of a stimulus on behavior was often modulated by various “states” of the animal, for example, those corresponding to food or water deprivation. The construct of motivation, both championed (Lorenz, 1950) and rejected (Hinde, 1960) in ethology as a device for explaining the generation and organization of behavior, served critical, but fairly proscribed, roles in experimental psychologists’accounts for learning and action. First, motivation was often thought to act as a performance variable energizing behavior at the time of action. Issues that attracted investigation included the specificity of motivational states in modulating behavior (e.g., do motivational states irrelevant to the task solution influence behavior?; Kendler, 1946), whether motivational states would energize behavior in the absence of explicit eliciting cues for that behavior (Sheffield & Campbell, 1954), and whether the energizing properties of motivational states could come to be controlled by external stimuli as a result of conditioning (Seligman, Bravman, & Radford, 1970). Although often framed in very different ways, these questions remain with us (Holland, 1991; Swithers & Hall, 1994).

Second, the establishment of associations was often thought to require the operation of some motivationally based reinforcement process to serve as a catalyst for, or to stamp in, stimulus-response (S-R) associations. The nature of this reinforcement process was the subject of great theoretical debate and spanned a range of possibilities including both the reduction (C. L. Hull, 1943) and the induction (Sheffield & Roby, 1950) of drive states (e.g., hunger, thirst, pain, fear, frustration), as well as the occurrence of consummatory behaviors. The 1950s saw the performance of a variety of experiments designed to test the reinforcement powers of events that, for example, reduced drives but failed to elicit consummatory behaviors (e.g., the delivery of food directly to the stomach or blood stream) or vice versa (e.g., the use of tasty but noncaloric foods, or sham feeding). The issue persists, albeit with altered terminology and purpose; for example, Myers and Hall (2000) found that the sensory and postingestive properties of sucrose can serve different roles in reinforcing Pavlovian conditioning in rats.

By the 1960s, emphasis shifted to concern for the interplay of learning and emotional processes. According to twoprocess theorists (e.g., Mowrer, 1947; Rescorla & Solomon, 1967), a major role of Pavlovian conditioning was the conditioning of emotional responses (CERs). These CERs were manifested not only in characteristic motor and autonomic responses, but also in the modulation of other, ongoing behavior, including learned operant behavior and unlearned consummatory behavior. Fear conditioning—in which, for example, tone-shock pairings endow the tone with the ability to elicit freezing or crouching responses and heart rate changes, to suppress lever pressing for food reward and drinking for its own sake, and to enhance responding to avoid shocks (see Mackintosh, 1974, for examples)—remains one of the more popular preparations for the study of conditioning and its neurobiological bases.

Learning and Temporal Contiguity

Early theories of learning agreed that the formation of associations was critically influenced by time, especially the temporal arrangement of the CS (or an operant response) and the US. Early work with eye-blink conditioning and other preparations suggested that conditioning occurred only when the CS/response occurred slightly before the US, on the order of a few seconds at most. The optimal CS-US interval was described as approximately half a second, with a rapid drop in the rate or amount of conditioning obtained with shorter or longer intervals. However, by the 1960s the most popular laboratory for Pavlovian conditioning procedures (see Mackintosh, 1974, for examples) routinely used CS-US intervals that were one or two orders of magnitude greater (10–100s). Indeed,the greatest blow to the claim that strict temporal contiguity of CS and US was critical to conditioning was struck by Garcia, Erving, and Koelling (1966), who showed that flavor aversion learning, whereby animals learn to reject flavors that are paired with the induction of illness, occurs readily over intervals measured in hours.

Figure 17.1 shows a set of graphs in which some performance measure is plotted as a function of the CS-US interval in a variety of conditioning preparations. Clearly, there is no best interval for conditioning; rather, different conditioning preparations reveal different parameter spaces. This observation is consistent with Lorenz’s claim that characteristics of learning must be highly species specific and task specific. Nevertheless, it is notable that despite the substantial variation in the absolute time intervals over which those functions apply, the functions are remarkably similar in form across a range of preparations. Each is bitonic, with conditioning best at intermediate values,declining rapidly with shorterintervals and declining more slowly with longer intervals.

Another important early finding about the effects of time on conditioning was that the interval between conditioning episodes (the intertrial interval, or ITI) is important as well. Generally, conditioning is facilitated by longer ITIs (Gormezano & Moore, 1969); indeed, the aphorism that spaced practice is better than massed practice survives as a principle in education.

Recent research has indicated a more complex relation between the ITI and the CS-US interval. In many conditioning preparations the CS-US interval function is modulated by the ITI such that the effectiveness of any given CS-US interval in producing conditioning depends on the ITI (Gibbon, Baldock, Locurto, Gold, & Terrace, 1977). Specifically, the ratio between the CS-US interval and the ITI is often a better predictor of conditioning than is either interval alone (for illustrations, see Gallistel & Gibbon, 2000; Holland, 2000; Rescorla, 1988a).

Cognitive Reformulations of Conditioning

Modern thinking about associative learning has taken a different track, following the lead of classical association theory rather than of reflexology. Most contemporary theorists describe conditioning as “the learning of relations among events so as to allow the organism to represent its environment” (Rescorla, 1988b, p. 157). Within this perspective, operant and classical conditioning are models of animals’ learning of relations between their own behavior and environmental events, and among environmental events out of their control, respectively. In Tolman and Brunswik’s (1935) terms, they model processes whereby animals become sensitive to the “causal texture” of their environment (p. 43).

This description differs in two important ways from earlier ones. First, it stresses a more abstract view of learning, dispensing with the primacy of transfer of control of reflexes or stamping in of habits. Learning a relation between a metronome and food might permit transfer of a salivary reflex controlled initially by the food, but it might also produce a range of other behavioral changes. Quantitative models within this perspective seldom relate learning directly to the performance or probability of a response, but instead are couched in terms of constructs like associative strength, signal value, and expectancy. Although these models assume that these constructs are related in some lawful manner to animals’ behaviors, they typically voice little concern for the nature or function of the behavioral consequents of association. Behavior is often reduced to a necessary but occasionally inconvenient assay of underlying associative learning.

Second, these more cognitive descriptions of learning emphasize the construction of internal representations of events and their relations, which may then guide behavior. Instead of learning to perform particular behaviors because of an arrangement between various events, the animal is assumed to learn about the events and their relations. The consequent representational structure then may be used to guide behavior in a more flexible fashion than is accorded by the simple transfer of a reflex from one stimulus to another, or the attachment of a new response to a discriminative stimulus. As a result, more emphasis has been placed on the nature and richness of representational processes, in addition to the rules by which associations are formed.

For the most part, both of these trends have been salutary. The reformulation of the problems of associative learning not only has broadened the domain of inquiry and application of simple learning principles but also has brought learning and behavior theory into more fruitful contact with other branches of psychology. On the other hand, this reformulation has often been construed as leaving behavior itself out of the picture, further separating psychological and ethological approaches. Nevertheless, by freeing associative learning from the confines of the reflex tradition, cognitive reformulations opened the door for considering the behavioral products of learning from ethological perspectives. In a later section we describe several examples in which cognitive perspectives have been combined with interests in behavior and its functions.

Beyond Temporal Contiguity: Information and Contingency

As shown in Figure 17.1, strict temporal contiguity is not necessary for associative learning. Likewise, it is now clear that mere contiguity of two events is also not sufficient for associative learning; rather, in some sense one stimulus must provide information about the occurrence of the other. We illustrate this point with two important phenomena, Rescorla’s (1968) contingency effect and Kamin’s (1968) blocking effect.

Rescorla (1967, 1968, 1969a) found that the associative learning that developed from repeated pairings of a CS and a US depended on the probability of US presentation in the absence of the CS, in the ITI. Figure 17.2, panel A, shows a cartoon of two conditions in this experiment. In both conditions rats received the same number of US presentations during the CS. Those procedures differed, however, in the probability of US presentation in the ITI. In Group E the US was less probable in the ITI than during the CS, and in Group R the US was equiprobable in the presence and absence of the CS. Despite the identical numbers of CS-US pairings, only rats in Group E acquired a CR. Indeed, parametric studies showed that the asymptotic level of conditioning attained was a regular function of the probabilities of US delivery during the presence and absence of the CS (Figure 17.2, panel B), as would be expected if animals were calculating the correlation, or contingency, of CS and US.

The Kamin (1968) blocking effect may be the most widely studied example of the insufficiency of temporal contiguity for associative learning. In a basic blocking study two groups of animals each receive pairings of a compound stimulus with a US (e.g., tone + light → food). Prior to this compound training, animals in the Blocking treatment received pairings of one of the stimulus elements (e.g., the light) with the same US, whereas the animals in a Control condition did not. Prior conditioning of the light blocks conditioning to the tone: A test of responding to the tone alone at the end of the experiment showed considerably more responding after the Control treatment than after the Blocking treatment, despite identical conditioning experience with the tone.

In both the contingency and blocking effects, conditioning appears to depend not just on CS-US contiguity alone, but also on the amount of information the target CS provides about the occurrence of the US. When the US is equiprobable in the presence and absence of the CS or when the US is already perfectly predicted by another CS, the target CS fails to acquire conditioning. The development of quantitative learning theories to deal with phenomena like these has led to important advances in the understanding of associative learning. Interestingly, most of these theories have embraced these phenomena by reformulating the idea of contiguity so that it applies to mental events instigated by CSs and USs, rather than the events themselves. One class of theories emphasizes the role of past learning in modulating the effectiveness of reinforcers (USs), whereas another class focuses on changes in the ability of CSs to participate in associative learning.

Variations in the Effectiveness of Reinforcers: The Rescorla-Wagner Model

The Rescorla-Wagner (1972) model has been the most influential modern learning theory. Not only did it provide simple accounts for contingency, blocking, and other puzzling phenomena, but also it led to predictions of a large number of new phenomena, many of which were counter to the intuitions derived from previous conceptualizations of conditioning. Within this model the amount of learning that occurs on a conditioning trial is a simple function of the discrepancy between the expected and actual values of the reinforcer presented on that trial. More formally, V_A = _A1(₁ – V_A..Z), where V_A refers to the change in the associative strength (V) of the CS “A” on a given trial, _A and ₁ are constants that describe the rate of learning about the CS “A” and the US “1,” respectively, ₁ is the maximum amount of associative strength supportable by the US “1,” and V_A..Z refers to the aggregate (total) strength of all CSs (A through Z) present on that trial. The aggregate strength is obtained by a simple summation rule by which the strength of a compound of several elemental CSs is the sum of the strengths of those constituent elements, for example, V_AB = V_A + V_B.

The key to the Rescorla-Wagner model is that the efficacy of a US in establishing learning on a given conditioning trials depends not just on its intrinsic reinforcing value () but also on the extent to which that value is already anticipated as a consequence of CSs that signal it (V_A..Z). Thus, the effectiveness of a US as a reinforcer is modulated by prior learning. Consider first the course of simple acquisition of conditioning to a CS, A. Because the strength of CS_A is initially zero and there are no other CSs present, the amount of learning about CS_A on the first conditioning trial is large. As associative strength accrues to CS_A, the discrepancy between the actual value of the US () and its expected value (V_A..Z), and hence the increments in learning about CS_A, become proportionally smaller on each successive trial. Thus, this model anticipates the frequently observed, negatively accelerated growth function, or law of diminishing returns. The reinforcer is maximally effective when it is unexpected and gradually becomes ineffective as it becomes better predicted by the CS.

The observation of blocking follows just as simply. In the first phase of a blocking experiment the associative strength of CS_A, V_A, will approach . Thus, when the novel CS_B is compounded with CS_A in Phase 2, the US will already be well anticipated on the basis of CS_A; that is, the expression ( – V_AB) will be small. As a result, the US will be ineffective as a reinforcer, and CS_B will acquire little or no associative strength despite repeated CS_AB-US pairings. Likewise, CS_A will acquire little additional associative strength.

By contrast, the rats trained with the Control procedure enter Phase 2 with no conditioning to either CS_A or CS_B. Consequently, for these rats the US is an effective reinforcer at the beginning of Phase 2, allowing both CS_A and CS_B to acquire associative strength on each trial until the US is wellpredicted by the CS_AB compound, that is, when V_AB = . The amount that each element (A and B) acquires is a function of its intrinsic rate parameter, . If _A = _B, then each will acquire conditioning at the same rate, and the asymptotic strengths of CS_A and CS_B will be equal, V_A = V_B = 0.5. Recall that learning will cease when the US is perfectly anticipated, that is, when – (V _A+V_B) = 0. Thus, the Control rats acquire considerably more strength to the added CS_B than do the Blocking rats, an outcome that defines the occurrence of blocking.

At the same time, note that asymptotically the strength of CS_A is lower in the Control rats (V_A = 0.5), which received initial conditioning of CS_A in compound with CS_B, than in the Blocking rats (V_A = ), which received initial conditioning of CS_A alone. This observation of greater conditioning to a CS when it is separately paired with a US than when it is presented in compound with other cues defines another common phenomenon of compound conditioning, overshadowing. Within the Rescorla-Wagner model this phenomenon occurs because the US is rendered ineffective as a reinforcer before each individual element can reach .

In each of the previous examples, the discrepancy or error term (₁ – V_A..Z) ranged from 0, which supported no additional learning, to , which permitted maximum increments in associative strength. If, however, the aggregate prediction (V_A..Z) is greater than , this error term will have a negative value, and V_A will be negative. Within the RescorlaWagner model, this loss of associative strength is equated with the acquisition of an opposing tendency: conditioned inhibition. If V_A is driven below zero, CS_A is said to be a conditioned inhibitor. Notably, the conditions under which conditioned inhibition develops—overexpectation of the reinforcer (the aggregate prediction is greater than )—are complementary to those that are necessary for the establishment of excitation (the underexpectation of the reinforcer, when the aggregate prediction is less than ).

Unfortunately, when presented by itself, a stimulus with negative associative strength may not control behavior, and thus may be indistinguishable from a cue with no strength. As a result, a number of indirect tests of inhibitory conditioning have been used. The most common are summation and retardation tests (Rescorla, 1969b). In a summation test, a suspected inhibitor (CS_A) is presented in compound with a known exciter. By the Rescorla-Wagner summation rule

(V_AB = V_A + V_B), if V_A < 0, then V_AB will be less than V_B, so CS_A will suppress responding to CS_B. In a retardation test, the suspected inhibitor is paired directly with a US, and the course of excitatory learning is examined. If the stimulus initially possessed inhibitory strength, then it would need to first regain zero strength before showing acquisition of positive associative strength. Thus, relative to controls, acquisition of new excitatory learning would appear slower.

The integration of excitation and inhibition within a common, symmetrical framework permits the model to make some of its most counterintuitive predictions. For example, consider an experiment in which CS_A and CS_B are each first separately paired with a US. As a result V_A and V_B each will approach . Next, CS_A and CS_B are combined, and the CS_AB compound is again paired with the US. By the RescorlaWagner model, the aggregate prediction provided by CS_AB is now 2, whereas the US supports only . Consequently, pairing of the AB compound with the US results in losses of associative strength of CS_A and CS_B (Kremer, 1978; Rescorla, 1970). More surprisingly, if a novel CS_C is added to the compound along with CS_A and CS_B, losses in the strengths of all three stimuli will occur, again proportional to the s associated with those cues. Because CS_C was novel, loss in its associative strength would lead to its acquiring net conditioned inhibition. Thus, the same physical US may produce new excitatory learning when it is underexpected, no learning when it is perfectly predicted, and inhibitory learning when it is overexpected.

With an additional assumption, the Rescorla-Wagner model was also able to deal with the contingency data described earlier. That assumption was that the experimental context (e.g., the experimental chamber) itself could serve as a CS, like any other event, and hence could potentially modulate conditioning to explicit CSs. Rescorla and Wagner suggested that simple conditioning procedures could then be described as involving various discriminations between a compound of CS + Context and the Context alone. If the US is equiprobable in the presence of the explicit CS (CS + Context) and in its absence (Context), then a situation very much like blocking obtains, in which both a compound stimulus and one element of that compound are reinforced. According to the Rescorla-Wagner model, the explicit CS, like the added CS_B in blocking, should display little evidence of conditioning asymptotically. Not only did the model do an excellent job of predicting the asymptotic levels of conditioning to an explicit CS obtained with different reinforcement probabilities in the presence and absence of the CS (Figure 17.2, panel B), but it also described the trial-by-trial dynamics of acquisition (Rescorla, 1973b). Furthermore, the model correctly predicted that the CS should become inhibitory if the probability of the US was greater during the absence of the CS than in its presence. In that case, because the context alone acquires associative strength, presenting the CS + Context compound with no US (which cannot support conditioning and thus would have a  of 0) would produce an overexpectation of the US, eventually driving the strength of the explicit CS below zero.

In summary, Rescorla and Wagner (1972) described a simple model that both accounted for an array of otherwise puzzling data and provided a simple trial-by-trial mechanism for their occurrence. Perhaps most important, within this model the conditions for the development of excitation and inhibition are not the occurrence and nonoccurrence of physical events, as in earlier theories, but rather the under- and overexpectation of those events as a result of past learning.

The general notion of error-correction routines, by which the aggregate strength is adjusted to match that supportable by the reinforcer, has had a broad impact on behavior theory. For example, Wagner (1978) presented substantial evidence that the variations in processing of events depending on how well they are predicted on the basis of past learning goes beyond the reinforcement power of stimuli and includes their persistence in memory and their ability to elicit responses. Thus, a surprising event not only is a more effective reinforcer than is an expected event, but, ceteris paribus, also generates larger CRs and is more persistent in memory.

Problems With the Rescorla-Wagner Model

The Rescorla-Wagner model is not perfect. Miller, Barnett, and Grahame (1995) provided an overview of the strengths and weaknesses of this model; we mention four weaknesses. First, although the model gained considerable power by providing symmetrical conditions for the establishment and definition of excitatory and inhibitory learning and by placing excitatory and inhibitory associative strength along the same scale, there is considerable evidence against such symmetry. For example, within the model, presentation of a conditioned inhibitor (a CS with net negative associative strength) by itself should extinguish that inhibition, as the discrepancy between the expected negative value is followed by nothing, an event with a zero . But under most circumstances this procedure does not reduce the ability of the conditioned inhibitor to act in summation and retardation tests (e.g., Zimmer-Hart & Rescorla, 1974). Likewise, much data support the claim that the loss of conditioned responding when a previously trained CS_A is no longer followed by the US (extinction) involves not just the reduction inV_A as claimed in the model, but rather the acquisition of a parallel inhibitory structure, maintaining much of the original excitatory learning (Rescorla, 1993). Second, although the Rescorla-Wagner model attributes blocking, overshadowing, and related phenomena to variations in the acquisition of associations, some evidence suggests that they may instead be related to failures in the retrieval of associations (e.g., Miller, McKinzie, Kraebel, & Spear, 1996; but see Holland, 1999). Third, there is ample evidence that the summation assumptions of the RescorlaWagner model are unrealistic. Recent data (e.g., Rescorla, 2000) show that apportionment of changes in associative strength among the elements of compounds depends on the training history of those elements, not just on their saliences (s). Furthermore, it is often simplistic to treat a compound stimulus as no more than the sum of its elements; we discuss some aspects of this notion of configuration later. Finally, although in the interests of simplicity Rescorla and Wagner (1972) assumed —the rate parameter for learning about a CS—to be constant, there is compelling evidence that  can vary as a function of experience (e.g., Dickinson & Mackintosh, 1978; Rescorla & Holland, 1982).Indeed,as noted in the next section, many theorists attempted to account for phenomena like blocking by positing learned variations in processing of the CSs, rather than of the US.

Variations in Processing of Conditioned Stimuli

Another class of conditioning theories attributes variations in conditioning in blocking, overshadowing, and related procedures to variations in processing of the CSs, rather than of the US. These models are often termed attentional models because the learned alterations in CS processing can be described as learning to direct attention toward or away from particular stimuli, so that certain stimuli are “selected” for controlling action or acquiring learning in blocking-like tasks.

The earliest attentional models relied on the notion of a limited attentional resource to account for stimulus selection effects. For example, Sutherland and Mackintosh (1971) assumed that the acquisition of a CS-US association is accompanied by increased attention to that CS. To the extent that attention is directed to that stimulus, less attention is available for learning about other CSs. As a result, in a blocking experiment an animal fails to learn about the added B stimulus because all of its CS processing resources are consumed by A, leaving no opportunity for B to be associated with the reinforcer. Thus, within this approach, blocking occurs because the added CS is not effectively processed in contiguity with the US.

Subsequent attentional models explored other origins for alterations in effective processing of CSs. For example, Mackintosh (1975) suggested that animals evaluate the ability of each individual CS to predict the US on each conditioning trial, increasing attention () to the more predictive cues and decreasing attention to the less predictive cues. In a blocking experiment, prior training of CS_A makes it an excellent predictor of the US. Because the added CS_B is a relatively poor predictor of the US, the animal rapidly learns to ignore it (i.e., reduces its ), so little is learned about it.

Perhaps the most successful approach to variations in processing of CSs is that described by Pearce and Hall (1980). They posited that attention to CSs is a function of how surprising the US is:  is directly related to the absolute value of the Rescorla-Wagner error term, | – V_A..Z|. Thus, the presentation of a surprising US maintains or enhances the ability of CSs to enter into new associations (), whereas  is driven low when the US is well predicted. Within this theory, the addition of CS_B when the US is already well predicted in a blocking experiment results in the loss of CS_B so that little CS_B-US learning can occur. Likewise, if the US is changed (such that _{2   1}) when CS_B is introduced, then CS_B will remain high, allowing CS_B to be associated with the US. Notably, consistent with much data (e.g., Pearce & Hall, 1980), either increases or decreases in  will maintain CS_B within this model. By contrast, within the Rescorla-Wagner model, although increases in  would permit additional learning about CS_B, decreases in  would result in inhibitory learning about CS_B. Thus, the observation of this “unblocking” phenomenon when the US is replaced by one with a lower  has frequently been cited as evidence for enhancements of CS processing. In a later section we show how a combination of behavioral and neurobiological investigations has provided evidence for key portions of these claims.

Representation of CSs: Elemental and Configural Views

Psychologists have taken a number of approaches to how compound CSs are represented. At one extreme, models like the Rescorla-Wagner model describe compound CSs as simply the sum of their elements. This elemental description worked well in characterizing early conditioning data. Nevertheless, there is ample evidence that animals frequently treat compound stimuli as very different from their elements. Acommonly cited example is that negative patterning (sometimes called exclusive-or) discriminations are often learned very readily. In these discriminations, CS_A and CS_B are each reinforced when presented alone, but nonreinforced when presented in compound (CS_AB). Clearly, no simple summation rule can predict that the strength of a compound of two cues will be less than that of either one alone.

At the other extreme, Pearce (1987) suggested that all stimuli are unitary or configural and cannot be decomposed into separable elements. Thus, training a negative patterning discrimination is in principle no different from training any other discrimination. At the same time, this approach recognizes that a compound may generalize considerably to stimuli that might otherwise be described as its elements, than to other stimuli. This approach has fared remarkably well in predicting the outcomes of a variety of compound conditioning experiments (Pearce, Aydin, & Redhead, 1997), although other data clearly favor more elemental views (Rescorla, 1997).

Other descriptions of compound stimulus processing borrow from both extremes. For example, Rescorla suggested a “unique stimulus” account, in which an AB compound stimulus is described as embracing both the explicit A and B elements and also a perceptually generated configural cue unique to the compound. Within this view, the unique cue acts like any other stimulus and thus is conceived of as just one more stimulus element within a compound. Rescorla (1972, 1973a) showed how the addition of a unique cue to the Rescorla-Wagner model permitted that elemental model to account for a number of compound conditioning phenomena normally thought to be outside its purview, including negative patterning discriminations.

In response to results from investigations of brain function, a number of theorists have suggested that the elemental and configural aspects of stimulus compounds are processed by different brain systems, and hence may follow different rules. For example, Rudy and Sutherland (1995) suggested that animals acquire both simple elemental associations and configural associations, but that under normal conditions the output of the configural association system suppresses that of the elemental system. In a more elaborate manner, Schmajuk and DiCarlo (1991) formulated a detailed quantitative neural network model in which stimulus elements form both simple associations with output units (the US) and associations with configural, hidden units, which are themselves associated with other stimulus elements and the output units. It is important that although in this model the simple and configural units compete for association in much the same way as specified by Rescorla, they are assumed to be anatomically and functionally distinct, and hence may follow somewhat different rules.

Representation of the Reinforcer

Within the dominant view of associative learning of the 1950s and 1960s, the reinforcer served primarily as a catalyst for the formation of S-R associations between the CS and a response (Figure 17.3, panel A), which Dickinson (1980) termed procedural learning. By contrast, most recent learning theories assume that animals learn about the events they associate, not just because of them.

From this modern perspective, classical CS-US pairings result in the formation of S-S associations between internal representations of the CS and US (Figure 17.3, panel A), which Dickinson (1980) described as declarative learning. By this view, the elicitation of CRs by a CS is mediated by its activation of a representation of the US, which in turn evokes those CRs. Evidence for this assertion comes primarily from reinforcer revaluation experiments, in which posttraining changes in the value of the US are spontaneously reflected in later CRs. For example, using rats, Colwill and Motzkin (1994) first paired an auditory CS with food pellets and a visual CS with liquid sucrose. Then, one of the reinforcers was devalued by pairing it with the toxin LiCl in the absence of either of the CSs. Finally, responding to the CSs was reassessed in the absence of either of the reinforcers. Rats showed a spontaneous loss in responding to the CS that had been paired with the devalued US, relative to responding to the other CS. Comparable results are observed when one of the reinforcers is revalued by a motivational manipulation, for example, selective satiation of the subjects on one food (Holland, 1988; Malkova, Gaffan, & Murray, 1997) or selective enhancement of the value of one of the reinforcers by inducing a specific motivational state (Coldwell & Tordoff, 1993; Rescorla & Freberg, 1978).

Analogous findings suggest that operant responding is also often mediated by activation of an internal representation of the reinforcer. For example, Colwill and Rescorla (1985) trained rats to perform one response for food pellets and another for liquid sucrose. Devaluation of the food pellets by pairing with toxin produced a spontaneous reduction in the frequency of the first but not the second operant response. Studies like these imply that animals can learn about the consequences of their actions (i.e., response-reinforcer associations), not just because of the rewarding consequences of those actions (i.e., S-R associations stamped in by the food reinforcer).

The results of other studies suggest that associatively activated representations of events may substitute for their referents in a variety of learning functions (see G. Hall, 1996, for a review). For example, Holland (1981a, 1990a) showed that rats could acquire an aversion to a food flavor if they were made ill in the presence of an auditory CS previously paired with that food. Likewise, presentation of an auditory signal for a particular food could substitute for the food itself in the extinction of a previously established aversion to that food (Holland & Forbes, 1982b) and in the overshadowing of learning of an aversion to another food flavor (Holland, 1983b). Furthermore, in many circumstances learned expectancies of particular events can control ongoing behavior (Holland & Forbes, 1982a; Trapold, 1970).

Representation of CS-US Relations: Occasion Setting

Holland (1983a) suggested that under some circumstances a CS acquires the ability to modulate the action of an association between another CS and the US. This modulatory power, often termed occasion setting, is typically studied with conditional discrimination procedures, in which the relation of one CS with the US depends on the presence or absence of another CS. For example, in a serial feature positive (FP) discrimination, a target CS is paired with food only when it is preceded by another feature CS (feature → target → food, target-alone → nothing). If the feature-target interval is sufficiently long, rats solve this discrimination by using the feature to set the occasion for conditioned responding based on target-US associations. By contrast, if the feature and target are delivered simultaneously on food-reinforced trials, rats instead form associations between the feature and the US, as anticipated by theories like the Rescorla-Wagner model.

Several kinds of evidence support a distinction between simple association and occasion setting (see Holland, 1992, and Swartzentruber, 1995, for reviews). Perhaps most convincing is the observation that the ability of a stimulus to act as an occasion setter is independent of any simple associations it may have with the US. For example, after serial FP training, repeated nonreinforced presentations of the feature alone typically have little lasting effect on its ability to modulate responding to the target cue, despite eliminating any CRs due to simple feature-US associations. A more dramatic example is found after feature negative (FN) discrimination training, in which the target is reinforced when presented alone but not reinforced when presented in compound with the feature CS. Analogous to FP discriminations, simultaneous FN discriminations establish inhibitory feature-US associations, whereas serial FN discriminations endow the feature with the ability to inhibit the action of the target-US association. After simultaneous FN training, direct featureUS pairings destroy the feature’s inhibitory powers, as measured by summation and retardation tests (Holland, 1984a). By contrast, after serial FN training, direct counterconditioning of the feature in this way often leaves intact (or even enhances) the feature’s ability to inhibit responding to the target (Holland, 1984a; Holland, Thornton, & Ciali, 2000; Rescorla, 1991). Thus, a negative occasion setter may at the same time elicit a strong CR and inhibit the ability of other CSs to elicit that CR.

Occasion-setting phenomena have been found in a variety of conditioning preparations, including, for example, autoshaped key pecking in pigeons, conditioned suppression of lever pressing in rats, rabbit eyelid conditioning, and drug discrimination training in rats (see Schmajuk & Holland, 1998, for examples). Not surprisingly, the conditions under which these phenomena are established, as well as the details of the phenomena themselves, vary from preparation to preparation. Nevertheless, the evidence suggests that modulatory functions of CSs are pervasive and substantial. Many investigators have suggested that experimental contexts, which may include spatial, geometric, and other features of the conditioning chamber, time of day, and so forth, are especially likely to play modulatory roles in conditioning (G. Hall & Mondragon, 1998; Holland & Bouton, 1999). Likewise, several researchers (e.g., Davidson, 1993; Holland, 1991; D. M. Skinner, Goddard, & Holland, 1998) have suggested that internal states, like hunger or thirst, may often act by modulating the effectiveness of other cues in eliciting learned or unlearned behaviors.

Researchers have proposed a variety of theoretical accounts of what is learned in occasion setting (see Holland, 1992; Schmajuk, Lamoureux, & Holland, 1998; Swartzentruber, 1995, for reviews). Holland (1983a) and Bonardi (1998) suggested that occasion setters involve hierarchical representation of specific event relations such that the occasion setter is associated with or modulates the activity of a particular CSUS association. Rescorla (1985) suggested that occasion setters act more broadly by modifying a threshold for activation of the US representation by eliciting CSs. A number of accounts for occasion setting relate it to configural learning more generally (e.g., Brandon & Wagner, 1998; Pearce, 1987, 1994). Within this approach, occasion setters and their targets are configured into a single unit that is distinct from the individual eventrepresentations. Each of these approaches captures a portion of the available data, but none is supported unequivocally. Perhaps the most comprehensive and detailed account for occasion-setting phenomena is a neural network model offered by Schmajuketal. (1998), which expands on the Schmajuk and DiCarlo (1991) model mentioned earlier by combining the modulatory and configural approaches.

The Representation of Temporal Relations

As noted earlier, within most theories of learning, the interval between CS and US was a critical variable in determining the rate or asymptote of learning. However, temporal intervals themselves were not thought to be represented by the animal: The only effect of arranging different temporal relations among events was the establishment of different associative strengths. These differences in associative strength were not distinguishable from those resulting from manipulation of any other variable, such as the amount of training or CS salience.

Contemporary research has shown that CSs also provide information about the time of US delivery. That is, animals learn not only because of the arrangement of temporal relations among events, but also about those relations (Gallistel & Gibbon, 2000; Gibbon & Church, 1990; Miller & Barnet, 1993; Savastano & Miller, 1998). The most obvious evidence for this assertion comes from studies of response timing. In many conditioning preparations, both operant and classical, the magnitude of conditioned responding increases systematically, exhibiting a peak near the time of US delivery (Figure 17.4, panel A). This temporal distribution of responding within the CS-US interval often displays what is known as the scalar property (Gibbon, 1977): Its variance is proportional to its mean. When normalized proportionally to the CS-US interval, the resultant distributions are identical, regardless of CS-US interval (Figure 17.4, panel B). There is currently a great deal of interest in the psychological mechanisms of timing in these circumstances (see Higa, 1998, for a review).

Evidence for timing of US delivery exists even in conditioning situations in which there is no clear-cut temporal gradient of responding. For example, Miller and his colleagues have found that a variety of conditioning phenomena that involve the addition of new stimuli to previously trained CSs in a serial fashion, such as blocking and second-order conditioning, depend on the contiguity of the added stimulus and the expected time of the US provided by the trained cue, rather than on the temporal relation between the CSs themselves (e.g., Savastano & Miller, 1998).

Synthetic Approaches to the Study of Learning in Animals

Although modern associationism’s emphasis on signal value, associative strength, and so forth seems a far cry from ethological concerns with the functions and origins of behavior, much current research in learning within each of these traditions borrows liberally from the other.This research combines experimenter-centered methods and theoretical constructs from experimental psychology with more animal-centered concerns with natural function and evolution from ethology. Furthermore, the infusion of the methods, interests, and orientations of neuroscience into both arenas has provided additional common ground.

Many synthetic trends can be identified. Researchers from the ethological tradition have been quick to adopt the procedures and analytic tools of experimental psychology to their purposes.Moresignificantly,manyoftheinterests,theoretical constructs, and terminology of experimental psychology have been imported into ethologically oriented research endeavors. For example, the face of behavioral ecology has gradually changed such that a great deal of research, in the field and in the laboratory, has concerned evolutionary and adaptive aspects of memory, representation, and cognition (see Shettleworth, 1998, for extensive treatment of these issues). The study of optimal foraging illustrates this trend. Field observations about food selection have given way to complex models of behavior that have converged to a large degree with related work being conducted by experimental psychologists (e.g., Kacelnik & Bateson, 1997; Kacelnik & Krebs, 1997).

Likewise, experimental psychology has been changed by the more animal-centered approach of ethology. In their study of what they construe as basic psychological processes, experimentalpsychologistsincreasinglyhaveselectedmoreecologically valid tasks and have been more open to questions about the adaptive significance of the behavioral systems that they study. For example, researchers interested in memory processing in rats are abandoning standard auditory-visual tasks in Skinner boxes for spatial learning, odor-guided food selection, and even social learning tasks, which are more characteristic of problems that rats face in nature. Of course, the downside of this ecumenicalism is that we abandon well-controlled (and well-studied) preparations in favor of those that we know little about and that give us less control. But a reasonable response to that problem is simply to take the time to uncover the basic characteristics of these new tasks and to refine them in ways that make greater experimental control possible.

Today, psychologists are more likely to recognize that behaviors sampled in conditioning experiments may be embedded in more extensive behavioral systems, which have been shaped by the demands of the niches in which they evolved. Often, the determinants and characteristics of learning may be better predicted from those perspectives than from any other. Consider two examples. Earlier we mentioned that the optimal CS-US interval differed dramatically across conditioning preparations, a few hundred milliseconds for eyeblink conditioning, several seconds for aversive cardiac conditioning, and tens of minutes for flavor aversion learning. What psychological principles account for these differences? Although there have been attempts to deal with these differences without reference to questions of function (Krane & Wagner, 1975), functional considerations have provided greater insight, or at least simpler rules of thumb. A shock to the eye provokes an eyelid response, but also a number of autonomic responses that may be related to behavioral flight systems. An eye blink is a useful response to a signal that the eye will be insulted within a few hundred milliseconds, but not to a signal that damage may occur in a few seconds. By contrast, heart rate changes in preparation for flight are useful with a warning of a few seconds, but not a few hundred milliseconds. Likewise, a flavor aversion learning mechanism that spans only seconds is unlikely to evolve in animals challenged by slow-acting food toxins. Thus, the particular effective range of interval values may vary on a species- or system-specific basis. At the same time, as noted earlier, the form of interval functions seems conserved more generally.

Another example of the value of considering laboratory tasks from an adaptive perspective is the classic case of cueto-consequenceselectivityintheaversiveconditioningofrats. Garcia and Koelling (1966; see also Domjan & Wilson, 1972) found that rats readily associated flavors and toxin-induced illness, as well as auditory-visual stimuli and shock-induced pain, but were poor learners of the other combinations, flavorpain and auditory-visual-illness. Although no simple psychological process predicts this selectivity, it is obvious from a consideration of problems faced by rats in nature. Rats normally select foods (which might make them ill, but are unlikely to cause peripheral pain) primarily by flavor but are unlikely to taste things that are about to cause them pain. Comparative studies, using animals that select food on different bases, support the simple view that animals are better able to solve tasks that are more like the ones they face in nature (e.g., Garcia, Lasiter, Bermudez-Rattoni, & Deems, 1985). Although many psychologists’ first reactions to these kinds of findings could be characterized as either defiant or apocalyptic, others were quick to recognize that the ease of associating any two items in conditioning might depend on existing, intrinsic relations between those stimuli (see Rescorla & Holland, 1976). This recognition spilled over into exclusively psychological realms; stimuli related by Gestalt grouping principles such as similarity (Rescorla & Furrow, 1977), spatial proximity (Testa, 1975), and partwhole relations (Rescorla, 1980) all are more readily associated in conditioning studies than are stimuli not sharing those relations.

At the same time, analysis of apparently unique, specialized examples of learning often reveal contributions of more general learning processes. For example, as noted earlier, current research suggests that imprinting shares many features with other examples of single event learning. Furthermore, Hoffman and Ratner (1973) suggested that imprinting may be profitably analyzed in the context of Pavlovian conditioning, in which associations are formed between initially neutral aspects of the imprinting stimulus and stimulus features that are critical to the initial following responses, as in Pavlovian conditioning. These associations allow the initially neutral aspects of the imprinting stimulus to elicit following responses and to serve as conditioned reinforcers. It is important to recognize that this learning brings the birds in frequent contact with these stimuli, rendering them less likely to elicit fear-withdrawal responses that are typically generated by novel stimuli. As a result, later filial approach behavior is controlled by a variety of perceptual aspects of the imprinting stimulus, but fear-withdrawal responses, which compete with filial behavior, are controlled by stimuli other than the imprinting stimulus. In support of these claims, Hoffman and his colleagues demonstrated the acquisition of these associative functions by neutral stimuli in imprinting situations. Furthermore, they found that adult filial behavior may even be induced to nonimprinted stimuli if the competing fear-withdrawal responses are habituated extensively by forced exposure to those stimuli later in life. They argued that in typical studies of filial imprinting the bird flees from nonimprinted test stimuli and so is never given the opportunity to learn about them. Thus, they interpreted some cases of the apparent time sensitivity and irreversible nature of imprinting as the result of general features of associative learning and habituation, which may be general to a number of species, including primates. Although it is unlikely that Hoffman and Ratner’s (1973) account provides anything near a complete account of imprinting, it provides a valuable perspective: Even examples of learning that show apparently unique properties may share more general characteristics.

It is only fitting to note that the study of biological mechanisms of behavior has also been a powerful trend bringing experimental psychology and ethology together. Indeed, the dividing line between neuroethology and behavioral neuroscience may be fainter still (see Moss & Shettleworth, 1996, for examples). Neuroscience has provided shared methods as well as a general reductionist approach that is less put off by perceived differences in the nature of the hypothetical constructs of the two fields. Furthermore, in some cases, common mechanisms of plasticity seem to underlie examples of learning that seem radically different on the surface. For example, developmental plasticity in the cortex associated with visual experience and adult plasticity in the hippocampus resulting from the induction of long-term potentiation both involve excitatory glutamate, especially the N-methyl-D-aspartate (NMDA) receptor. Indeed, the NMDA receptor is widely involved in many forms of learning and plasticity in a wide variety of species (e.g., Bliss & Collingridge, 1993; Brown, Kairiss, & Keenan, 1990; Constantine-Paton, Cline, & Debskie, 1990).

We do not mean to claim that there is a single synthetic approach to the study of learning, but rather a range of approaches, some drawing more from ethology and others more from experimental psychology. In the remainder of this section we present three case studies that exemplify some of this range. Each analysis has profited from appreciation of both ethological and general process perspectives, as well as an interest in neurobiological mechanism. The first, a study of learning and memory processes in bird song, comes primarily from the ethological end of the continuum; the second, a study of rats’learning to anticipate food, is from the opposite end; and the third, a study of sexual conditioning in quail, falls toward the center. We do not offer them as the best available examples; rather, they reflect our own interests and research. Recent research on the foraging (e.g., Kacelnik & Krebs, 1997) and food caching (e.g., N. S. Clayton & Soha, 1999; Shettleworth & Hampton, 1998; Suzuki & Clayton, 2000) of birds and rodents provide other particularly compelling examples.

Learning and Memory Processes in Bird Song

Dialects and the Early Study of Song Learning

As early as the seventeenth and eighteenth centuries, aviculturists in both Asia and Europe realized that vocal behavior in songbirds is remarkably labile and could be manipulated by experience in ways that the vocalizations of other avian species could not (see Konishi, 1985; Thorpe, 1961; Welty & Baptista, 1988, for discussions of these early ideas). Although bird vocalizations can be influenced by experience, B. F. Skinner (1957) observed that the vocal behavior of nonhuman animals could not be easily manipulated by operant and classical conditioning procedures that are so powerful in modifying other motor patterns. Only recently have such experimental procedures proven to be at all effective in modifying the learning of conspecific vocalizations in birds (e.g., Adret, 1993; Manabe & Dooling, 1997). This limitation suggested that the processes involved in vocal learning could be distinct in many ways from those mediating at least some other learned responses.

The manner in which song is learned was first investigated experimentally by ethologists who were interested, in part, in understanding the origins of intraspecific variation in bird song. The song of all songbirds possesses species-typical attributes that allow one to distinguish one species from another based purely on the song (Ball & Hulse, 1998; Catchpole & Slater, 1995; Searcy & Andersson, 1986). The fact that vocalizations are species typical is true for probably all bird species and indeed for most species that produce complex vocalizations, including various insect and other vertebrate groups such as anurans (Ball & Hulse, 1998; Searcy & Andersson, 1986). However, unlike most of the species in these other groups, in several songbird species, systematic geographic variation was found within this species-typical pattern. Thus, by listening closely to a song one could identify both the species singing and the place of its origin. Studies of two oscine species, the chaffinch (Fringilla coelebs) in England and the white-crowned sparrow (Zonotrichia leucophyrs) in the United States, were especially important in establishing the fundamental principles governing the development of bird song. Both chaffinches (Marler, 1952) and white-crowned sparrows (Marler & Tamura, 1964) exhibit such marked geographic variation in their songs that the suggestion arose that these species possess something akin to human dialects. Marler and Tamura (1964) observed that white-crowned sparrows in Marin County, California, could be distinguished from those living around Berkeley, which in turn could be distinguished from those living around Sunset Beach, based on variation in the end phrasing of the song. Although all these birds produced a song that is clearly recognizable to a trained listener as whitecrowned sparrow song, the birds also exhibited systematic variation within this song that allowed trained listeners to identify their geographic origin.

Such within-specific variation in song behavior raised questions concerning the origins of song. Does this variation represent genetic variation as had been observed for many morphological traits that vary geographically within a species (Mayr, 1963), or does it result from differences in learning? Thorpe (1958), working with chaffinches, and later Marler (1970), working with white-crowned sparrows, employed methods referred to by the early ethologists as the Kaspar Hauser approach. That is, birds were raised in isolation, especially acoustic isolation. These studies demonstrated unequivocally that song is learned. Birds raised in acoustic isolation produce abnormal songs never heard in nature, and these never improve with age. Birds that heard taperecorded song early in life later developed songs that were species-typical in their structures. This discovery suggested that something akin to cultural transmission occurs in birds.

Of the over 9,000 living bird species that are divided into some 23 orders, vocal learning has been demonstrated in 3 of the orders: the songbird order just described (passeriformes), the order psittaciformes (parrots and related species), and the order trochiliformes (i.e., hummingbirds; Baptista 1996; Baptista & Schuchmann, 1990; Farabaugh & Dooling, 1996). Most experimental work has been conducted on songbirds, and that is our focus here. The songbird order can be divided into two suborders, the oscines (suborder passeres)andthemoreprimitivesuboscines.Inadditiontothe morphological differences between oscines and suboscines that the taxonomists have identified, there appear to be qualitative differences in vocal development between these two taxa. Suboscine vocalizations are not learned (Kroodsma, 1988; Kroodsma & Konishi, 1991), and suboscines do not require access to auditory feedback to develop normal vocal behavior. Although this generalization must be considered with some caution, given that few representatives of suboscine families have been studied, these data do suggest that an important qualitative shift in the mechanisms mediating the development of vocal behavior occurred when the oscine passerines evolved as an independent taxonomic group. To conclude, there is no evidence that suboscine species learn their song, but oscine species clearly do.

The Process of Song Acquisition

Since the pioneering work on white-crowned sparrows and chaffinches, studies of many species of oscine songbirds have all found that song is learned (e.g., Kroodsma, 1982; Kroodsma & Baylis, 1982; Marler, 1991; Thorpe, 1958). There are many species-specific and intraspecific variations in song learning among oscines (e.g., Baptista, 1996; Kroodsma, 1996; Marler, 1987, 1991; Marler & Nelson, 1993; Slater, 1989; West & King, 1996; West, King, & Duff, 1990). A variety of approaches have been taken to the study of song learning; in some cases laboratory studies are conducted with the obvious advantage of experimental control, and in other cases field studies are conducted with the obvious advantage of sensitivity to the many environmental factors that impinge on the developmental process. In most cases the focus is on male birds. It is impossible in this brief review to discuss fully the diversity in developmental processes that has been identified. However, some useful generalizations have emerged. Our goal is to highlight important findings that reflect both the laboratory and the field approach. Many important concepts have emerged from studies of bird song that are relevant to a consideration of animal learning as well as human language learning and production such as the idea that many birds are critical- (or sensitive-) period learners and that songbirds have innate predispositions to learn their own species song (suggesting that they possess something akin to the language acquisition device described by Chomsky, 1965), and that learned songs may be overproduced and then selected for enduring production in adulthood based on the consequences of social interactions (Marler, 1997).

In some species, called age-independent learners, the ability to learn and modify song may be retained throughout life, whereas in other species this propensity may be age limited (Marler, 1987; Slater, Jones, & Ten Cate, 1993). Among agelimited learners, the learning process can be usefully divided into a sensory phase and a sensorimotor phase (Marler, 1987). In the sensory phase nestlings hear conspecific vocalizations either from their fathers or from nearby males and form an auditory memory (or sensory template) of some sort that represents these songs. Conspecific songs heard relatively early in development tend to be remembered better than songs heard later, indicating that there is a critical or sensitive period during which auditory memories that guide subsequent song production are most effectively formed (Nelson, 1997). This phenomenon is reminiscent of the postulated critical period learning associated with imprinting or language acquisition; it has been claimed that the ability to easily learn new languages closes off at the onset of adolescence in humans (Lenneberg, 1967; Newport, 1990). In a similar vein, in a variety of songbirds males can acquire songs from remarkably little exposure to song early in life (Nelson, 1997). For example, just 10 to 20 presentations are sufficient for normal song learning in nightingales (Luscinia megarhynchos; Hultsch & Todt, 1989) and 30 repetitions on a single day in song sparrows (Peters, Marler, & Nowicki, 1992). Thus, songbirds have superb abilities when it comes to memorizing, and later retrieving from memory, songs they have heard early in life.

During the sensorimotor phase birds reaching puberty (usually during the first spring after they are hatched) first start producing very soft unstructured vocalizations that are referred to as subsong. These sounds have often been compared to the babbling of prelinguistic infants. These birds then shift to louder, more stereotyped vocalizations in which rehearsal of previously learned song begins, containing many elements that can be recognized as adult song. This is called plastic song. Finally, they develop an adult crystallized song that matches the auditory memory formed earlier (Marler & Peters, 1982a). The process is schematized in Figure 17.5.

This entire process is very dependent on auditory feedback. If a bird is deafened after the occurrence of the sensory phase but before the sensorimotor phase, normal song will not develop (Konishi, 1965, 1985). This indicates that auditory feedback is necessary for the final crystallized song to be properly matched to the song previously memorized during the sensory phase. In some species, such as the zebra finch, it was long thought that once song has crystallized into its adult form, auditory feedback was no longer necessary for song maintenance—as if a motor tape of some sort were in place after crystallization that could maintain the song independent of feedback. However, a study of deafened adult male zebra finches (Taeniopygia guttata) revealed deficits in the song of deaf males, suggesting that auditory feedback is required throughout life for the maintenance of this stereotyped behavior (Nordeen & Nordeen, 1992).

Depending on the species, the sensory phase and the sensorimotor phase may overlap with one another or may be separated by several months, during which the song memory from the sensory stage is stored and retained without any evidence of overt practice. Recent evidence suggests that in species in which there is a substantial interval between the sensory phase and the sensorimotor phase, significant processing of song information occurs in that interval. Male white-crowned sparrows usually have a hiatus of approximately 5 months between the two phases of song learning, but if they are treated with testosterone they can be induced to sing crystallized song months before they would normally exhibit this behavior (Whaling, Nelson, & Marler, 1995). Nevertheless, the song produced by these birds is abnormal, resembling songs produced by sparrows kept in acoustic isolation (Whaling et al., 1995). This finding suggests that the storage phase is not one of passive retention, but that it contains processes that play a significant role in song learning (Whaling et al., 1995). However, it is also possible that the vocal production apparatus had not fully matured by 5 months and that this is why the premature song sounds abnormal.

The Auditory Memory That Guides Song Learning

What characterizes the auditory memory that is formed during the sensory phase, and is its formation constrained in any way? At least two important ideas relevant to the study of human language and other areas of cognitive psychology have emerged from studies of this memory for song. One concerns the concept that the process of memory formation involves selective or guided learning in that species-typical vocalizations appear to be privileged: They are learned preferentially. A more controversial idea concerns the degree to which memory encodes species-typical patterns of vocal behavior that exist before a bird even hears the song of its own species. The experimental basis of both of these ideas is well illustrated by studies of song sparrows (Melospiza melodia) and swamp sparrows (Melospiza georgiana) conducted by Marler and colleagues (see Marler, 1987, for review). These two species are age-limited learners that look alike, and they are members of the same genus and therefore are closely related taxonomically. However, their songs are easily distinguished. Although these species are often raised within earshot of one another, they tend to form memories only for their own species song. This has been verified by laboratory studies, suggesting that the formation of auditory memories is somehow directed such that conspecific songs are preferentially memorized (Marler, 1987). This phenomenon is also suggestive of certain processes that occur during language learning associated with the closure of the critical period for language acquisition in humans. For example the closure of the critical period is thought to involve, among other processes, a loss of the sensitivity to phonetic contrasts from nonnative languages (reviewed in Jusczyk, 1997). In other words, by the time we reach adulthood, humans have a perceptual selectivity not only for human language but also for their own native language.

When raised in acoustic isolation, the abnormal song that is produced retains species-typical attributes such as the number of notes per song and the number of trilled syllables per song (Marler & Sherman, 1985). Even though this isolate song is clearly abnormal, one can still tell apart the isolate song of the two species based on the song’s acoustic structure. These are among the data that have led Marler and Nelson (1992) to postulate the somewhat radical hypothesis that the vast majority of information about song may be preencoded in innately specified brain circuitry. They argued that auditory experience primarily selects what is to be preserved and later produced as crystallized song from what is to be discarded and never produced later in life. According to this view, then, the postulated auditory memory that guides later song development may be largely specified at birth. This idea remains controversial, but the argument parallels in many ways discussions among cognitive scientists concerning the nature of innate representations that specify the predisposition to learn language in humans (see Chomsky, 1980; Elman et al., 1996; Fodor, 1983; Pinker, 1994).

Recent experiments with white-crowned sparrows support the notion that at least certain aspects of song structure are pre-encoded in the auditory memory guiding song learning (Soha & Marler, 2001). Sparrows were presented with a variety of phrase models that lacked species-typical structure (or syntax). Even though this was not present in the stimulus presented to the sparrows to be copied, the birds successfully copied the model phrases and assembled them into a speciestypical structure.

Social Effects on Song Learning

It is important to note that both the sensory and sensorimotor phases of song learning are labile to a certain degree and can be influenced by various types of social and auditory experience. It has been known for some time that the social milieu in which a bird develops can influence the type of song that is eventually crystallized and produced in adulthood. Behavioral interactions of various sorts have been described that influence both the sensory phase and the sensorimotor phase of song learning. For example laboratory studies demonstrated in white-crowned sparrows that the presence of a live bird that tutors a developing juvenile can apparently extend the sensitive period during which auditory memories are formed (Baptista & Petrinovich, 1984), and the number of songs learned by juvenile European starlings (Sturnus vulgaris) is significantly greater when they listened to nearby conspecific male tutors as compared with tape-recorded songs (Chaiken, Böhner, & Marler, 1993). This evidence suggests that either the nature of the auditory memory that is formed is influenced by the social milieu that the bird grows up in or that a particular set of social interactions tends to select songs already in memory that will be produced later in life (see Baptista & Gaunt, 1997, and Nelson, 1997, for contrasting views).

There are data supporting the view that behavioral interactions during the sensorimotor phase influence what songs already in memory will be crystallized and produced by adults. In several species it has been demonstrated that males learn (or have in their memories) more songs during the sensory phase than they will later crystallize and produce as adults (Marler & Peters, 1982b). In field sparrows (Spizella pusilla) interactions even during the sensorimotor period between a male and his neighbors influence which songs will be selected for crystallization and inclusion into the final repertoire (Marler & Nelson, 1993; Nelson, 1992). Songs most similar to the neighbor’s songs are retained, whereas those that are different are rejected. It is as if a process akin to operant conditioning influences what songs are selected for later crystallization among those in auditory memory.

Marler and Nelson (1993) referred to this process as action-based learning to contrast it with memory-based learning, which refers to cases where crystallized songs are produced in reference to previously formed memories independently of social interactions during the sensorimotor period. This process of the adult vocal milieu influencing vocal development parallels observations made about babbling in infants. Infants are thought to tune their babbling based on the adult language environment they experience (BoyssonBardies & Vihman, 1991). Thus the adult vocal environment shapes vocal development in humans as well as in songbirds. In any case, in many species the selection of songs overproduced seems to be highly influenced by the fact that some songs match those being produced locally and others do not. This result indicates that reinforcing aspects of the social interaction lead to the selection of certain behaviors, a notion very reminiscent of Thorndike’s concept of selective learning processes in instrumental conditioning. The details of this process remain to be worked out (e.g., how much matching is required for a song to be selected). But again, as we saw with imprinting, at least certain aspects of song learning include processes in common with more general learning processes.

In summary, it is clear from this cursory review that social interactions can influence both the sensory and the sensorimotor phases of song learning. Much remains to be learned about the structure of the auditory memory possessed by developing birds and the degree to which it is formed or modified by experience. Under certain circumstances early experience can have radical effects on the nature of the auditory memory (Baptista, 1996). How these social experiences are able to override the natural limitations on song learning is unknown.

The Neurobiology of Bird Song

One of the most appealing aspects of the study of bird song learning is that a neural circuit has been described in the songbird brain that represents a neural specialization that has evolved in association with the ability to learn, produce, and perceive complex vocalizations (Brenowitz, Margoliash, & Nordeen, 1997; Nottebohm, Stokes, & Leonard, 1976). Thus it is possible to investigate in detail the neural basis of the vocal learning process. For this reason, the neural and endocrine basis of the production, learning, and perception of bird song has emerged as an active area of research in the last 25 years. It is impossible to review in detail the many discoveries that have been made about the relationship between brain and behavior as it relates to song. We describe the basics of the neural system that mediates song behavior and highlight a few findings that demonstrate how the study of bird song learning has influenced studies of the neurobiology of song.

When he began his studies on the neural control of song, Nottebohm noted that the fact that vocal learning requires that motor output be guided by auditory input has important implications for how the neural circuit controlling of song and song learning would be organized (Nottebohm, 1980; Nottebohm et al., 1976). He hypothesized that there should be close connections between the motor system controlling song and the auditory system. This reasoning generated a series of lesion and tract-tracing studies in canaries that resulted in the discovery of the song system (Nottebohm, 1996). Subsequently, comparative studies of the song control circuit with other birds clearly indicate that there are several neural features associated with vocal learning and production that are unique to songbirds (Ball, 1994; Brenowitz, 1997; Kroodsma & Konishi, 1991). In particular, studies of canaries (Serinus canaria) and zebra finches have revealed a well-defined vocal control circuit that includes a group of interconnected, distinct nuclei that differ in form and structure in the telencephalon, mesencephalon, and brain stem (Bottjer, Meisner, & Arnold, 1984; Nottebohm, Kelley, & Paton, 1982; Nottebohm et al., 1976;Wild,1997).Thetelencephalicportionofthiscircuitappears to be a neural specialization that occurs only in the oscine brain (Brenowitz, 1997; Kroodsma & Konishi, 1991). The song control circuit includes motor nuclei that are involvedin song production and nuclei that also exhibit auditory characteristics that appear to be involved in the auditory feedback necessary for vocal learning (for reviews see Brainard & Doupe, 2000; Konishi, 1989; Margoliash, 1997; Nottebohm, 1993; Nottebohm et al., 1990). The motor pathway that is necessary for the production of song in adult birds consists of a series of nuclei that control the neural output to the vocal production organ, the syrinx (Nottebohm, 1993; Nottebohmetal., 1976). This motor pathway is illustrated in Figure 17.6. Three key nuclei in this pathway are the high vocal center (HVc), the robust nucleus of the archistriatum (RA) and a portion of the motor nucleus of the XIIth cranial nerve (nXIIts). These three nuclei form a serial projection (HVc to RA to nXIIts). Efferent projections from motor neurons in nXIIts innervate the syrinx. Song is produced when the muscles associated with the two separate sides of the syrinx are activated, leading to a change in the configuration of the membranes of the syrinx (Nowicki & Marler, 1988; Suthers, 1997). Projections from RA also innervate brainstem structures that coordinate song production with respiration (Wild, 1997). Lesions to nuclei in this motor pathway have profound effects on song production (e.g., Nottebohm et al., 1976). In particular, lesions to HVc cause deficits in vocal production reminiscent of Broca’s aphasia in humans. Electrophysiological studies of neurons in HVc, RA, and nXIIts reveal that their activity is synchronized with singing behavior (McCasland, 1987; Vu, Mazurek, & Kuo, 1994) and that these nuclei are hierarchically organized (Yu & Margoliash, 1996). Nuclei RA and nXIIts are myotopically organized (i.e., organized in relation to syringeal musculature; Vicario, 1991;Vicario & Nottebohm, 1988); however, HVc is not (Fortune & Margoliash, 1995; Yu & Margoliash, 1996). This motor pathway of the song control system thus appears to coordinate temporal and phonological aspects of song.

As illustrated in Figure 17.6, neural impulses from HVc also reach RA through a more circuitous route than the motor pathway (Bottjer, Halsema, Brown, & Miesner, 1989; Okuhata & Saito, 1987). This anterior forebrain pathway consists of a projection from HVc to area X of the lobus parolfactorius (LPO), which in turn projects to the medial portion of the dorsolateral nucleus of the anterior thalamus (DLM). DLM projects to the lateral portion of the nucleus magnocellularis of the anterior neostriatum (lMAN), which in turn projects to RA (Figure 17.6). In adult zebra finches all of the song control nuclei in this anterior forebrain pathway contain cells that are tuned to respond to the presentation of species-specific song (Margoliash, 1997; Solis & Doupe 1997). Auditory information is conveyed to the song system via connections between the telencephalic auditory area, Field L to areas adjacent to HVc and RA (Fortune & Margoliash, 1995; Kelley & Nottebohm, 1979; Vates, Broome, Mello, & Nottebohm, 1996; see Figure 17.6). Related areas for auditory processing include the caudomedial neostratium (NCM) and the caudal ventral hyperstriatum (cHV; see Figure 17.6). These areas are also connected to HVc and RA (Vates et al., 1996) and have been shown to contain cells that express immediate early genes specifically in response to conspecific song (D. F. Clayton, 1997; Mello, Vicario, & Clayton, 1992), and they have electrophysiological properties that are similarly somewhat selective to conspecific song (Stripling, Volman, & Clayton, 1997). Therefore, these auditory areas may be the part of the neural circuit that is specialized to process conspecific vocalizations.

The anterior forebrain pathway has been implicated in many feedback effects on song production and in the process of song learning. For example, lesions of nuclei in the forebrain pathway of zebra finches (especially area X and the lMAN) before species song is crystallized result in song abnormalities, such as lack of note stereotypy and abnormal song length (e.g., Bottjer et al., 1984; Scharff & Nottebohm, 1991). Similar lesions made after the development of stereotyped song have no effect on song production in adult male zebra finches, although there are effects of such lesions on the maintenance of song in adult white-crowned sparrows (Benton, Nelson, Marler, & DeVoogd, 1998). The neural mechanisms mediating the feedback effects between song production and the stored auditory memory are still not well understood, but many properties of the anterior forebrain pathway provide intriguing hints about how this might work. Song selective neurons, present throughout the anterior forebrain pathway, respond more strongly to the bird’s own song and sometimes to the tutor song (see Brainard & Doupe, 2000, for a review). These cells are clear candidates for at least part of the neural system mediating the comparison between the stored memory and auditory output (Brainard & Doupe, 2000). Interestingly, these responses are strongly gated by the behavioral state of the animal. They are absent or much weaker in anesthetized or sleeping birds (Dave, Yu, & Margoliash, 1998; Schmidt & Konishi, 1998). The significance of this gating remains to be clarified. Brainard and Doupe (2000) speculated that the motor act of singing itself may open the gate so that the feedback effects can be exerted. What is clear is that there are neural specializations in the songbird brain that can be directly related to the auditorymotor interface required for song learning to occur.

Neural changes in the song circuit responsible for the closing of the sensitive period for song learning have yet to be definitively identified (Nordeen & Nordeen, 1997; Nottebohm, 1993). However, developmental changes in the connections between the anterior forebrain pathway and the efferent motor pathway (i.e., the lMAN to RA projection) have been the focus of attention in this regard. What is known (reviewed in Nordeen & Nordeen, 1997) is that there is a dramatic decline in the density of NMDAreceptors in male zebra finches in lMAN coincident with both the memory acquisition and sensorimotor phase of song learning. Furthermore, the pharmacological blockade of these receptors does impair song learning, and therefore the NMDA mechanisms that have been implicated in many forms of learning and developmental plasticity are important in the song learning process as well. However, these developmental changes in the NMDA receptors do not close the sensitive period. Manipulations of the timing of the sensitive period either by social isolation or hormone treatment do not change the timing of the decline in NMDA receptors.

The neural basis of sensorimotor learning is very poorly understood. However, as mentioned previously, in many species this learning process involves a selection of song based on social interactions. One intriguing idea is that ascending catecholamine projections to the song system (Appeltants, Absil, Balthazart, & Ball, 2000) are activated by these social interactions and contribute to these selectivity. Immediate early gene induction in the anterior forebrain pathway is profoundly influenced by the social context of song production in male zebra finches (Jarvis, Scharff, Grossman, Ramos, & Nottebohm, 1998). It has been hypothesized that these social context effects on gene expression are regulated by catecholamine projections to the song system (Jarvis et al., 1998). If this is the case, it is also reasonable to postulate that the consequences of song matching on the selection of songs during the sensorimotor phase of song learning could also be regulated by these pathways.

Food-Related Appetitive Learning in Laboratory Rats

As with many animals, a major part of the activity of rats is the procurement and consumption of food. As noted earlier, since Wallace Craig’s (1918) work both ethologists and experimental psychologists have distinguished between these appetitive behaviors and the more stereotyped, consummatory aspects of feeding. Although Craig’s view was that appetitive behavior was more easily modified by experience, it is clear that both components are subject to modification by learning. Casually speaking, it is advantageous to learn when and where food is available, what kind of food it is, and how much of it there is. Learned signals for food can aid procurement by orienting the animal to the proper location at the right time. But they also can provide the animal with useful information about the ingestive (e.g., flavor) and postingestive (e.g., nutrient absorption) consequences of the expected foodstuffs.

A Behavior Systems Approach to the Anticipation of Food

In traditional approaches to conditioning, the form of Pavlovian CRs is thought to be determined by the choice of the US, which provides the reflexes that can be transferred to the CS. A behavioral systems approach to food-based learning goes beyond this observation by adding the ethological perspective that, besides the simple reflexes elicited by delivery of the particular food US, the animal’s learned behavior potentially includes a range of more complex, preorganized behavior systems that are normally involved in the procurement and consumption of food. Both the dynamics of learning itself and the expression of learning in behavior will depend on how the learning task contacts these systems. These points of contact include not only the nature of the US but also the signals (CSs) used, the structure of the local environment (context), and more global conditions including the animal’s deprivation and housing conditions.

The power of a behavioral systems approach to understanding food-related associative learning in rats is perhaps best illustrated by the work of Timberlake and his colleagues (e.g., Timberlake & Silva, 1995; see Fanselow, 1994, and Fanselow & Lester, 1985, for comparable discussions of defensive behavior). Timberlake proposed that a variety of individual action patterns related to the procurement and consumption of food is hierarchically arranged within a variety of modules, modes, and subsystems, each of which has its own characteristic elicitors and terminators, as well as timing and sequential properties. For example, Timberlake described a set of actions such as pouncing, grabbing, biting, and gnawing that are organized within a capture module, which in turn is part of a focal search mode within predation. Most important, learning may occur at many points and levels within the system. Within-module learning may result in changes in the stimulus control, frequency, or timing of particular actions, whereas learning that occurs at the supermodular level may alter the relation between whole sets of behaviors organized within each of those modules or modes. Thus, the behaviors displayed in conditioning experiments will vary depending on which of these units are accessed and altered as a function of the events and relations involved in those studies. Within this approach most conditioned behavior may be viewed as the result of the modulation of existing behavioral patterns by associative learning, much as twoprocess theorists thought CSs modulated ongoing instrumental behavior.

Consider the effects of using as CSs events that are unusual from an experimenter-centered view but obvious from an animal-centered perspective. Timberlake, Wahl, and King (1982) examined the effects of signaling food delivery with a ball bearing that rolled across the chamber floor about 5 s in advance of the food. The rats acquired a sequence of behaviors including chasing the bearing, seizing it, and handling and gnawing it as if it were a food item. These behaviors were not acquired if the rolling ball bearing was unrelated to food delivery, and thus they were the consequence of associative learning. Timberlake described this learned behavior as reflecting the incorporation of existing predatory modules (wild rats chase and consume moving insects) and food-handling modules into a sequence of behaviors culminating in food consumption. It is interesting that similar experiments with a range of rodent species that show varying degrees of insect predation revealed a correlation between their natural predation and the likelihood of ball-bearing chasing and mouthing. Likewise, the form of the conditioned behavior varied with the CS-US interval in a predictable manner. With moderate intervals, the rats engaged in the complete sequence described earlier, reflecting focal search and related modes, but with shorter intervals the food-handling module dominated, and the rats often deposited the bearing in the food cup and gnawed on both the food and the bearing (Silva & Timberlake, 1998). Furthermore, Holland (1984b) found that when a CS is presented after the US (backward conditioning), it often acquires normal postconsummatory behavior such as grooming (see also Silva & Timberlake, 2000).

Timberlake and his colleagues also examined rats’ learning within social modules. In one study (Timberlake & Grant, 1975) food delivery was signaled by the arrival of either a restrained live rat or a rat-sized block of wood.Although subject rats oriented to both signals, when the signal was a ratsized block, their conditioned behavior was dominated by approach and gnawing on the block; but when the signal was another rat, they displayed a variety of social approach and contact behaviors characteristic of normal food-sharing and solicitation activities. Critically, these behaviors were far more frequent when the signaling rat was paired with food delivery than when it was explicitly unpaired with food. Thus, the enhanced social behavior was a consequence of associative learning. Furthermore, the nature of this behavior was consistent with normal patterns of adaptive behavior. For example, Timberlake (1983, 1994) reported that only adult signals of food supported these conditioned behaviors, consistent with observations that adult rats use the food preferences and consummatory behaviors of other adult rats, but not of juveniles, to guide their food selection. Likewise, when water rather than food was used as the US, these behaviors did not appear, consistent with observations that rats’food selection involves social modules, but water consumption does not. Finally, comparative studies with a variety of rodent species found correlations between the extent of food sharing and the conditioning of these kinds of social behaviors.

Although this approach is unquestionably rich in potential, there has been relatively little study of neural and behavioral mechanisms involved in these examples. A more simplistic experimenter-centered approach, in which significant US events like food, water, or electric shock are signaled by various simple visual or auditory CSs, has been profitable in addressing these issues. In line with reflexological expectations, the form of conditioned behavior is dependent on the choice of US: Rats freeze during these CSs when they signal shock, and they approach and contact the delivery cup when they signal food or water deliveries, often gnawing or licking the cup as appropriate (e.g., Holland, 1979). However, as in Timberlake’s studies, it is clear that the choice of CS is also important in determining CR form. For example, diffuse auditory cues paired with food typically elicit a quick jump or startle response, followed by agitated head movements, often directed toward the site of food delivery. By contrast, localizable, overhead visual cues (illumination of lamps) paired with food usually first causes the rat to rear on its hind legs, often orienting toward the light source, followed by a more passive contact with the food delivery cup (Holland, 1977). Furthermore, consistent with Timberlake’s scheme, if the light source extends significantly into the experimental chamber, the rat is likely to contact it with its paws and mouth as well (Holland, 1980b), integrating more of the normal feeding sequence into the learned behavior.

As in Timberlake’s studies, temporal variables are important determinants of response form in this conditioning situation. Not surprisingly, the influence of CS characteristics on CR form is greatest close to CS onset, and the effects of US features are most obvious closest to the normal time of US delivery (Holland, 1979, 1980a, 1980b). Observations that the form of CRs is influenced by the CS-US interval have important implications for the study of the role of time in conditioning. Holland (1980a) found distinct CS-US interval functions for each of the several CRs observed when visual and auditory CSs are paired with food. Depending on the choice of CR measured, the optimal CS-US interval for learning the same tone-food or light-food relation could be identified as 1 s, 5 s, 10 s, or 30 s. Furthermore, although some of those CRs were exquisitely timed to occur maximally near the time of US delivery and exhibited the scalar property, others, like rearing, were mostly confined to the few seconds near CS onset, and still others occurred maximally near the middle of the ISI. As Timberlake pointed out, signals with different temporal relations with the US are likely to contact different modules of food-related behavior. Thus, the experimenter’s choice of a behavioral measure of association could lead to vastly different conclusions about the relation of time and association. In fact, data like those of Holland and Timberlake suggest a multiplicity of timing functions, each tuned to a behavioral system, rather than the more popular view of a single estimate of the time of US delivery (Meck & Church, 1982).

All of these results are consistent with a simple view that conditioning produces behavioral adaptation to the altered significance of the CS as well as to the upcoming US. Learning-dependent changes in processing of the CS may be of many sorts. Holland (1977) suggested that the simplest of these may be the conditioning-dependent potentiation of orienting behaviors controlled by the CSs prior to conditioning. Some of the CS-specific behaviors described earlier are also elicited by the CSs prior to their pairing with the US, but they rapidly habituate if the CSs are not paired with food. The potentiation of these initially unconditioned orienting behaviors enhances the likelihood that the rats’behaviors will be guided by the best signals for important events.

Considerable evidence indicates that these CS-dependent products of associative learning are separable from those appropriate to the upcoming US. As noted earlier, the temporal determinants and characteristics of CS- and US-dependent behaviors are often reciprocally related. Furthermore, several experimental manipulations have different effects on the two classes of behaviors (Holland, 1990b).

Finally, a neural systems analysis also supports this distinction. Brain pathways involved in the potentiation of orienting behavior to CSs paired with food are not critical to the acquisition and display of US-dependent behaviors. Gallagher and her colleagues (Gallagher, Graham, & Holland, 1990; Han, McMahan, Holland, & Gallagher, 1997) found that the conditioned potentiation of orienting behavior depends on the integrity of connections between the central nucleus of the amygdala and dorsal striatal regions. Although rats with interruptions in this circuitry were unaffected in their acquisition of US-appropriate food-related behaviors, and in their unconditioned orienting to the CS prior to conditioning, they failed to acquire conditioned orienting with CS-US pairings. Thus, this amygdalar circuitry seems to be critical for the potentiation of existing orienting responses (ORs) to events that have acquired significance through associative learning. Similar neural systems analysis might also be profitably applied to other module-level aspects of conditioned responding, such as those described byTimberlake and his colleagues, to understand better the roles of associative learning in the organization of behavior.

An important question in this kind of research is the extent to which the various CRs that occur under different circumstances indicate independent learning at the module level, for example, within separate neural pathways, and to which they all draw on common learning at some higher level. Despite the independence of brain systems for the learning of CSspecific ORs and food-related CRs, considerable evidence indicates that CSs that elicit differently formed CRs when paired with food nevertheless have a common associative basis at some level. For example, compound conditioning phenomena like blocking, unblocking, and second-order conditioning with simple lights and tones occur largely without regard to the specific CRs controlled by the CSs, suggesting that various CSs have access to properties of the US that determine these effects (Holland, 1977). Answers to these kinds of questions will lead us to better understanding of the organization of behavior through learning.

A Biobehavioral Analysis of Alterations in CS Processing

As noted earlier, there is considerable evidence that experience with a CS can affect its associability, that is, its ability to participate in new learning. Pearce and Hall (1980) proposed that CS associability was adjusted depending on how well predicted its consequences were. Expected outcomes reduce CS associability, whereas surprising events (unpredicted by the stimuli present on that trial) maintain or enhance it. Thus, CSs that consistently predict their outcomes will be less associable than will those that are relatively poor predictors of their consequences. This claim is reasonable from an adaptational perspective: It is more critical to be able to learn about cues whose significance has heretofore been unknown than about cues whose significance is already well established. From that same perspective, however, it is important that behavior be controlled by cues whose consequences are well known. Within the Pearce-Hall model, a CS’s associability () appears only in equations that specify new learning about that CS, and not in those that describe its control of behavior, which is a function only of its associative strength (V_CS) and its intrinsic salience (s_CS). Thus, the Pearce-Hall model distinguishes between controlled attention for new learning and automatic attention for action. Recent studies of brain pathways involved in appetitive conditioning support these claims and provide some novel insights into the nature of the behavioral processes involved in the modulation of CS associability and learning.

Early behavioral studies that tested the Pearce-Hall model were designed to evaluate contrasting predictions of that model from those of its major alternative, the RescorlaWagner model. Consider first the widely described case of unblocking. As noted earlier, learning about one element of a compound CS paired with a US is blocked by prior training of another element of that compound with the same US. Within the Rescorla-Wagner model, that blocking occurs because the US is already well predicted by the first CS when the second element is introduced, and hence is ineffective as a reinforcer. By contrast, within the Pearce-Hall model, blocking occurs because the presentation of a well-predicted US causes the associability of the added CS to be adjusted rapidly downward, minimizing learning about that cue.

In an unblocking procedure the value of the US is changed when the target stimulus is added. If this new value is greater than the original value, both theories predict conditioning of the added cue (unblocking). Within the Rescorla-Wagner model the positive discrepancy between actual and expected US values supports further excitatory learning, and within the Pearce-Hall model that discrepancy maintains the associability of the added CS long enough for it to acquire associative strength as a result of its pairings with the new US. However, if the US value is reduced when the new CS is added, the two models make opposite predictions. Because the discrepancy between actual and anticipated events is negative, it should establish negative (inhibitory) associative strength to the added cue according to the Rescorla-Wagner model. But within the Pearce-Hall model both positive and negative discrepancies enhance or maintain CS associability. Thus, the added CS will acquire excitatory associations with the new (smaller) US. Although the determinants of learned behavior in the case of such downshifts in US value are complex, excitatory learning, as predicted by Pearce and Hall, is often observed (Holland, 1988).

Holland and Gallagher (1993b) and Bucci, Holland, and Gallagher (1998) found that disruption of a brain system, including amygdala CN and the substantia innominata/nucleus basalis and its projections to the posterior parietal cortex prevented unblocking with downshifts in the US value. Other experiments showed that these brain system manipulations also interfered with several other phenomena attributed by Pearce and Hall to surprise-induced associability increases (Han, Holland, & Gallagher, 1999; Holland, Chik, & Zhang, 2001; Holland & Gallagher, 1993a; Holland et al., 2000). At the same time, those manipulations had no effect on simple conditioning, on other measures of sensitivity to the change in US value, on unblocking observed with upshifts in US value, on blocking itself, or on a number of phenomena usually attributed to decreases in CS associability. From the results of a series of selective lesion experiments, Holland and Gallagher (1999) concluded that surprise engages this system, which selectively enhances attentional processing of CSs in a manner consistent with the Pearce-Hall model.

Parallel studies indicated that disruption of a cholinergic septal-hippocampal circuit had complementary effects on decreases in CS associability. These manipulations (hippocampal lesions or cholinergic deafferentiation of hippocampus by immunotoxic lesions of the medial septal area and the vertical limb of the diagonal band) had little effect on phenomena attributed to enhancement of associability but interfered with a variety of phenomena indicative of losses of associability (Baxter, Holland, & Gallagher, 1997; Han, Gallagher, & Holland, 1995).

Holland and Gallagher (1999) concluded that increases and decreases in CS associability, as specified by the PearceHall model, are modulated by two distinct brain systems. Furthermore, some evidence suggests that these systems act independently and additively; most effects of combined destruction of both systems are predictable from the effects of separate lesions (Baxter, Bucci, Holland, & Gallagher, 1999). Moreover, neither system seems critical to the occurrence of phenomena that are easily described without reference to changes in CS processing, including simple conditioning and phenomena attributed to changes in US processing as claimed by the Rescorla-Wagner model (Holland & Gallagher, 1993a, 1993b).

We have taken advantage of this relative independence to parcel out the contributions of attentional and other processes to a variety of behavioral phenomena that may have multiple determinants. For example, Baxter, Gallagher, and Holland (1999) examined the effects of medial septal lesions, which interfere with associability decreases, on blocking. If blocking is the result of losses in associability of the added CS, as specified by the Pearce-Hall model and other attentional theories, then this lesion should reduce blocking. In a test of responding to the added cue, Baxter, Gallagher, et al. (1999) found no effect of the lesion. However, subsequent learning tests showed the associability of that cue to be greater in the lesioned rats, as would be expected if the lesion interfered with losses in associability in the blocking procedure. Baxter, Gallagher, et al. (1999) concluded that in intact rats, although the blocking procedure did produce reductions in CS associability, those losses were not responsible for the blocking effect, which instead was the result of processes specified by the Rescorla-Wagner model. If a predicted US is ineffective as a reinforcer from the very beginning of the blocking phase, then variations in CS processing that occur over the course of that phase are irrelevant to conditioning at that time.

Likewise, we exploited the CN lesion to help us understand a puzzling discrimination finding reported previously. Rescorla (1991) found that pigeons learned a serial negative patterning (NP) discrimination of the form feature CS → food, target CS → food, feature → target → no food more rapidly than a serial FN discrimination, feature → food, feature → target → no food. Although the latter discrimination can be solved simply by the acquisition of excitation to A and inhibition to X, solution of the patterned discrimination should be more difficult from a variety of perspectives. However, the relative ease of learning of the patterned discrimination might be anticipated within the Pearce-Hall model. In the NP discrimination, X’s associability should be enhanced because X is a relatively poor predictor of both Aand the US. By contrast, in the FN discrimination, X is a reliable predictor of the occurrence of Aand the absence of the US, and thus X’s associability should decline. By this view, elimination of surprise-induced enhancements of X’s associability by the CN lesion should eliminate the superiority of the NP discrimination learning. Holland et al. (2000) found just this outcome. Furthermore, the CN lesion did not affect learning in variations of the patterning procedure in which there was less opportunity for surprise-induced associability enhancements in normal rats.

In both cases, reduced preparations (Teitelbaum & Pellis, 1992) were used to understand better the nature of learning in the intact preparation. Thus, the study of biological bases of learning can contribute importantly to study at more purely behavioral or psychological levels. Holland and Gallagher (1999) described the use of similar strategies for informing other aspects of behavioral study as well.

Associative Learning and Motivational State

A key to the behavioral systems approach is the idea that associative learning alters behavior by modulating the activity of behavioral systems, which themselves may be organized within larger scale systems. As noted earlier, it has long been presumed that CSs can control a range of behaviors indirectly by modulating motivational systems or states. Thus, a CS paired with food may acquire the ability to influence a broad range of food-motivated behavior. For example, light-food pairings appear to endow the light with food’s reinforcing power. In Pavlovian second-order conditioning, subsequent tone-light pairings in the absence of food establish CRs to the tone (Hatfield, Han, Conley, Gallagher, & Holland, 1996; Holland & Rescorla, 1975), and in operant secondary reinforcement procedures, rats learn to press levers that result only in the presentation of that light (Everitt & Robbins, 1992). Likewise, a light paired with food can modulate the frequency of operant lever pressing for that food (Pavlovianto-instrumental transfer), perhaps by enhancing some motivational state that energizes appetitive behavior involved in food procurement (Dickinson, Smith, & Mirenowicz, 2000). Finally, that same CS can enhance consummatory behavior as well: Food-satiated rats eat more food in the presence of a CS for food than in its absence (Holland, Hatfield, & Gallagher, 2001; Weingarten, 1983; Zamble, 1973).

Although learning theorists have often suggested that these indirectly measured consequences of associative learning are most important because they modulate other kinds of learned behavior and emotional states (e.g., Mowrer, 1947; Rescorla & Solomon, 1967), for the most part they have been treated as just another index of abstract association formation. By contrast, within a behavioral systems approach, they may be quite independent, and hence interesting in their own right. Recent studies of brain systems suggest that these consequences of conditioning operations may be separately organized. For example, second-order conditioning, secondary reinforcement, and CS-potentiated feeding are all disrupted by lesions of the basolateral, but not central, amygdala (Everitt & Robbins, 1992; Gallagher & Holland, 1992;

Hatfield et al., 1996; Holland, Hatfield,  et al., 2001), whereas the opposite is true of Pavlovian-to-instrumental transfer (J. Hall, Parkinson, Connor, Dickinson, & Everitt, 2001). Moreover, neither lesion affects conditioned approach to the source of the food US (Everitt & Robbins, 1992; Hatfield et al., 1996). Recent evidence suggests that the amygdala’s role in these various phenomena involves the modulation of the activity of other brain regions that each organize different behavior systems (e.g., Petrovich, Setlow, Holland, & Gallagher, 2001; Setlow, Holland, & Gallagher, 2002).

Associative Learning and Representation of the Reinforcer

Earlier we noted that Pavlovian CRs often appear to be mediated by a representation of the US because posttraining devaluation of a US can spontaneously (and selectively) reduce responding to CSs paired with that US. It is interesting to note that response systems can be differentially sensitive to these devaluations. For example, Holland and Straub (1979) found a double dissociation between the effects of two ways of devaluing food and various classes of CRs. When food was devalued by toxin-induced illness, ORs to CS onset were unaffected, approach responses were moderately reduced, and responses related to food handling were substantially reduced. In contrast, when food was devalued by motioninduced illness, ORs were most affected, with much smaller effects on later-chain behaviors. Furthermore, food devaluation by satiation (Holland, 1981b) produced across-the-board (but selective) decreases. These data show that different behavioral systems make different use of information made available through learning.

Finally, it is worth noting that sensitivity of CRs to posttraining changes in the value of the reinforcer seems to depend on brain circuitry that includes the orbital frontal cortex and the basolateral amygdala (Baxter, Parker, Lindner, Izquierdo, & Murray, 2000; Hatfield et al., 1996; Malkova et al., 1997; Schoenbaum, Chiba, & Gallagher, 1998). Given the role of basolateral amygdala in many aspects of a CS’s acquisition of motivational significance (described in the previous section), it is interesting to speculate that information about sensory properties of the reinforcer, affected in devaluation procedures, and processing of motivational value, may converge in this region.

Sexual Approach Conditioning in Japanese Quail

A Behavior Systems Approach to Sexual Conditioning

As part of a long-term research program devoted to explicating the role of learning in the control of sexual behavior, Domjan and his colleagues (Crawford, Holloway, & Domjan, 1993; Domjan, 1994; Domjan, Akins, & Vandergriff, 1992; Domjan & Hollis, 1988) have developed a variety of behavioral procedures that are appropriate for the investigation of appetitive sexual behavior in quail. These methods include the application of Pavlovian conditioning procedures in which a neutral stimulus such as the illumination of a light or the presentation of a gray foam block serves as the CS that is temporally paired with access to a female that serves as the US. Male quail will quickly come to approach a CS when it is paired with the US in this way. This conditioned approach behavior shows common conditioning phenomena like blocking (Koksal, Domjan, & Weisman, 1994), conditioned inhibition (Crawford & Domjan, 1996), second-order conditioning (Crawford & Domjan, 1995), US devaluation effects (Hilliard & Domjan, 1995; Holloway & Domjan, 1993), context conditioning and context-specificity effects (Crawford, Akins, & Domjan, 1994; Domjan, Greene, & North, 1989), and potentiation of unconditioned sexual behavior (Domjan, Lyons, North, & Bruell, 1986).

As in Timberlake’s (1994) and Holland’s (1984b) studies, the nature of the conditioned sexual response depends heavily on the nature of the CS. Although quail will learn to approach just about any localizable CS, copulatory responses themselves are not acquired unless the CS includes plumage and other features of a female quail (Cusato & Domjan, 2000; Domjan, O’Nary, & Greene, 1988; Koksal et al., 1994). At the same time, however, CSs that support only approach behavior also have a profound facilitatory effect on copulation if they are presented along with a female quail, shortening the latency of initiation of copulatory behavior and enhancing sperm release (Domjan, Blesbois, & Williams, 1998). Furthermore, the topography of the approach and other CRs depends on the CS-US interval (Akins, Domjan, & Guitierrez, 1994), as in Silva and Timberlake’s (1998) and Holland’s (1980a) studies of rat feeding CRs.

During the course of these studies, Domjan and his colleagues discovered another phenomenon, which they referred to as a learned social proximity response. This response is a form of associative learning that is acquired by males after they have been allowed to copulate with females (Domjan & Hall, 1986a, 1986b; Domjan et al., 1986). In this procedure, a male is allowed to copulate with a female after the male has observed the female through a window in the test arena. The response involves a remarkable change in a male’s behavior: After a single copulation, males will spend the majority of their time (literally for days at a time) standing in front of the window and looking through it at the female (Domjan & Hall, 1986a, 1986b). The response is clearly associative in origin and depends on copulatory rather than visual experience with the female, because it will only be acquired by the male if he actually copulates with the female after seeing her (Balthazart, Reid, Absil, Foidart, & Ball, 1995; Nash, Domjan, & Askins, 1989). Domjan et al. (1992) argued that the learned social proximity response should be viewed as a type of associative learning in which stimuli from a female become directly associated with sexual reinforcement.

Although this social proximity response of male quail can be acquired to a variety of CSs, learning with a female quail CS is clearly privileged. For example, it is more prevalent with a live female quail CS than with various partial models, and stronger with a female quail CS than with a male quail CS (Nash & Domjan, 1991). Furthermore, Domjan et al. (1992) reported that this learned proximity response was acquired even when they introduced delays of 2 to 3 hr between the closing of the window providing a view of a female and the access to the female for copulation. Likewise, Koksal et al. (1994) found that model CSs that more closely resembled female quail visually were less susceptible to blocking by previously trained arbitrary CSs.

Domjan (1994) described the social proximity response as reflecting focal search within more extensive behavior systems in sexual conditioning. When isolated in a large arena, male quail will first pursue a general search for females, and then when a female is localized to a particular place, they exhibit a more focal search. Such variable searching behavior is characteristic of an appetitive behavioral response. As a good example of focal sexual search, the learned social proximity response in male quail thereby provides a useful measure of appetitive sexual behavior that one can contrast with the stereotyped sequence of the neck grab, mounting, and cloacal contact movements characteristic of the consummatory sexual response.

A Neuroendocrine Analysis of Sexual Conditioning

The social proximity response conditioning procedure has provided a very useful way to study the neuroendocrine mechanisms regulating different aspects of male sexual behavior, and in turn, the study of the hormonal and neural mechanisms has told us something about how naturally reinforcing stimuli exert their effects on learned responses. Social proximity response learning is relatively easy to obtain under laboratory conditions, but at the same time it involves aspects of the learning process that occur in male quail when they engage in sexual behavior in a natural context. Furthermore, it appears to provide a good indicator of appetitive sexual behavior, in that the male seems clearly to be engaging in this behavior in anticipation of copulatory behavior itself, but this learned behavior does not resemble consummatory copulatory behavior. Initial studies demonstrated that the presence of testosterone is required for the development and maintenance of this social proximity response and that this response is acquired only by males, not by females (Domjan, 1987; Domjan & Hall, 1986a, 1986b).

The studies described in this section used a modification of Domjan and Hall’s (1986a, 1986b) procedure to assess the neuroendocrine mechanisms mediating appetitive sexual behavior in quail to compare the mechanisms involved in the control of this aspect of sexual behavior with the relatively well characterized mechanisms mediating consummatory sexual behavior. In this modified procedure, quail are tested for relatively brief periods of time (25 min) so that a large number of subjects can be examined each day (Balthazart et al., 1995). All behavioral observations are carried out in an arena containing two compartments. The experimental male is tested by introducing him into the larger compartment that is separated by a sliding door from an adjacent smaller cage where the stimulus female is located. A small window is located in the middle of the sliding door and provides the male with limited visual access to the female. If the door between the female and male is opened and the male is allowed to copulate with the female, there is a radical change in his behavior. As in Domjan’s studies, males start to spend an inordinate amount of time (up to 95% of the 5-min period) in the area in front of the window. However, this learned social proximity response only develops if the male is permitted to copulate with the female. If the door between the two compartments is not opened, the response is never acquired. Similarly, if a given male fails to copulate during the first few tests, the acquisition of the proximity response is delayed until the first test when copulation will take place. These observations clearly indicate the reinforcing role of consummatory sexual behavior in the learning of the proximity response (Balthazart et al., 1995). This modification of the original procedure described by Domjan illustrates how a behavioral phenomenon can be modified to suit a more detailed mechanistic investigation while capturing the key features of the natural situation.

Domjan (1987) first demonstrated that the display of appetitive sexual behavior as measured by the learned social proximity response is dependent on the presence of high circulating levels of testosterone: If males are placed on short days that induce testicular regression and suppress testosterone secretion, they do not acquire this response. The steroid-dependence of appetitive sexual behavior was confirmed by a strategy of surgical castration that was combined in some subjects with a testosterone replacement therapy. Castrated subjects never learned the proximity response, but it was acquired by testosterone-treated castrates as well as by sexually mature adult males (Balthazart et al., 1995). The fact that this learned behavior is so dependent of the presence of gonadal steroids provides one with a good opportunity to study the neural mechanisms regulating the behavior. In males in many species including quail, testosterone is locally metabolized in the brainto either an androgen or an estrogen before it exerts its effects on male sexual behavior (Balthazart, 1989; Meisel & Sachs, 1994).By localizing in the brain both the relevant enzymes that metabolize testosterone to androgens or estrogens and the relevant receptors to which these hormones will then bind to exert their effects, one can gain valuable insights into the neural circuit regulating this behavior (Balthazart & Ball, 1998).

Therefore, more detailed studies of the neuroendocrine mechanism regulating this acquired response followed (Balthazart & Ball, 1998). Previous work in quail and in a variety of birds and mammals had demonstrated that the activation of copulatory behavior by testosterone requires its conversion via the enzyme aromatase into the estrogen 17estradiol (for a review, see Balthazart, 1989). It was therefore hypothesized that appetitive sexual behavior would similarly be dependent on estrogenic metabolites of testosterone. This was found to be the case based on three independent experimental approaches.

First, the learned proximity response could be acquired by castrated birds if they were systemically treated with either the endogenous estrogen 17-estradiol, or a synthetic estrogen, diethylstilbestrol (DES). These compounds were found to mimic the behavioral effects of testosterone (Balthazart et al., 1995). This idea was further supported by a second experiment that demonstrated that once the proximity response had been acquired, its expression could be blocked by daily injections of the antiestrogen tamoxifen. Although the inhibition of the appetitive proximity response was paralleled by a marked decrease in copulatory behavior as well, several aspects of the experimental protocol led to the conclusion that the disappearance of the proximity response was not mediated by the decreased copulatory behavior. One might argue, for example, that presentation of a female while copulation was inhibited might serve as a punisher for the proximity response. In this study appetitive sexual behavior was first measured in a series of behavioral tests during which the door separating the male and the female was never opened. In this situation, although males had no access to behavioral feedback of any sort that could indicate that their copulatory behavior was inhibited, they nevertheless exhibited a decline in appetitive behavior. Normally, the proximity response is maintained for long periods of time (over 20 tests during a 5-week period) when birds are tested in this extinction condition (door never opened and reinforcement no longer available). Therefore, the decline in appetitive behavior of the tamoxifen-treated males cannot be attributed to either simple extinction or punishment contingencies (Balthazart et al., 1995). It can therefore be concluded that maintenance of appetitive sexual behavior in quail requires the presence of estrogens.

The final confirmation of this conclusion was obtained in an experiment that assessed the effects of an aromatase inhibitor on the social proximity response that had been acquired by castrated birds treated with testosterone. The response was first learned by one group of castrated birds that received testosterone implants (CX + T groups). A second group of castrates received empty Silastic implants as a control manipulation (CX group); as expected, these birds did not learn the response. CX + T birds were then assigned to one of two subgroups that were matched based on the proximity response they had shown during the last two training tests. Beginning 4 days later, birds in one of the CX + T groups were injected twice a day with an aromatase inhibitor, and the other two groups received control injections. During the period when these injections were administered, appetitive sexual behavior tests were performed. In each series, the first four tests were performed under conditions of extinction (i.e., the door was not opened, so there was no access to the female), but free access to the female (door open) was provided during every fifth test. A specific and significant inhibition of the learned social proximity response was observed in these conditions, and this inhibition appeared to be independent of the accessibility or nonaccessibility of the female. This inhibition developed progressively during the period that the aromatase inhibitor was injected, and the inhibition was maximal during the last behavioral tests. Mean behavioral scores collected during the last three tests (when looking through the window was quantified in addition of the time spent in the test area) are illustrated in Figure 17.7. As can be observed, both responses indicative of appetitive sexual behavior were nearly completely blocked by the aromatase inhibitor.

In conclusion, these experiments demonstrate that appetitive aspects of male sexual behavior in quail are activated, as is the case for consummatory sexual behavior, by testosterone acting through its estrogenic metabolites. The endocrine specificity is therefore similar, if not identical, for the activation of both aspects of sexual behavior.

The learned social proximity response is not observed in female quail, even if they are treated with testosterone as adults. This finding suggests that the sex difference in the exhibition of the learned social proximity response is regulated similarly to the sex difference in consummatory sexual behavior (i.e., copulatory behavior) in quail, which is known to be organizational in nature. The term organizational means that the sex difference results from the exposure of female embryos to elevated levels of estrogens that causes the demasculinization of the neural substrate mediating the activation of sexual behavior (Balthazart & Ball, 1995). Males do not experience such high estrogen levels as embryos and develop a male-typical neural substrate mediating this behavior. Blocking aromatase activity by treating females on day 9 of incubation with an aromatase inhibitor prevents the demasculinizing effects of endogenous estrogen. This results in the females engaging in male-typical copulatory behavior in response to adult testosterone treatment (Balthazart & Ball, 1995; Balthazart, De Clerck, & Foidart, 1992; Foidart & Balthazart, 1995).

The role of embryonic estrogens in the sexual differentiation of learned social proximity response was tested by blocking estrogen synthesis in ovo. Control males and testosterone-treated females deprived of estrogens during embryonic life learned the social proximity response, but control females did not, despite the presence of high concentrations of testosterone in the plasma. The neural substrate mediating the learned social proximity response is therefore demasculinized by the action of embryonic estrogens during ontogeny, as is consummatory behavior.

Studies of the neural basis of this behavior have also been completed. The preoptic area in quail is known to express the enzyme aromatase as well as androgen and estrogen receptors, so it was an important candidate brain region as being part of the circuit regulating male-typical sexual behavior (Panzica, Viglietti-Panzica, & Balthazart, 1996). Previous lesion and hormone implant studies have implicated a sexually dimorphic nucleus in the preoptic region, the POM, as being necessary and sufficient for the activation of male-typical copulatory behavior (Balthazart & Surlemont, 1990; Balthazart, Surlemont, & Harada, 1992; Panzica et al., 1996). An analysis of the lesion sites within POM in relation to their effectiveness in knocking out different measures of appetitive and consummatory male sexual behaviors indicated that damage to the subdivision of the POM just rostral to the anterior commissure was the most effective in blocking copulatory behavior (Balthazart, Absil, Gérard, Appeltants, & Ball, 1998). Lesions to the POM inhibited the learned social proximity response (Balthazart et al., 1998). These marked effects of POM lesions on this measure of appetitive sexual behavior are somewhat surprising given suggestions in rodent species that the preoptic region is preferentially involved in the activation of copulatory performance and plays little or not role in anticipatory, motivational, or arousal aspects of male sexual behavior (Everitt, 1995; Liu, Salamone, & Sachs, 1997). Although the completeness of this dissociation between the preoptic region and the regulation of appetitive sexual behavior is probably an oversimplification (Baum, 1995; E. M. Hull, 1995; E. M. Hull, Du, Lorrain, & Matuszewich, 1997), these data do suggest that the involvement of the preoptic regions in both appetitive sexual behavior and consummatory sexual behavior may represent a significant difference in the regulation of male reproductive behavior in quail as compared to rodents and other mammalian species.

However, there is clear evidence of a dissociation in the brain areas regulating these two aspects of male behavior. In the case of the present study we have obtained the best evidence for a dissociation by investigating the significance of different subregions within the POM. Cells within the second and third rostro-caudal subdivisions of the POM seem to be particularly important for the control of appetitive aspects of male sexual behavior. At present, there is no obvious reason why this cell group would be preferentially important in the control of appetitive sexual behavior in male quail. Chemical neuroanatomical studies have not revealed any markers associated with neurotransmitter function that are enriched (orpresent at low levels) in this region as compared to adjacent parts of the POM (Panzica et al., 1996). Studies of cell activation involving the induction of immediate early genes or the measurement of 2-DG incorporation have not revealed precise subdivisions of the POM that correspond to the areas identified based on these lesions (Meddleetal., 1997). However, fos induction was higher in the caudal portion of the POM as compared to rostral portion in association with the occurrence of consummatory sexual behavior. Tract-tracing studies completed to date have not identified specificity in the connectivity of subregions of the POM, but these studies have not been designed to address this question properly. Future studies of variation in the hodological properties of these subregions of the POM are an obvious candidate for further investigation.

Conclusion

Overall, the study of animal learning is a much healthier enterprise due to the interactions among ethologists, psychologists, and neurobiologists. It is true that psychologists have often seemed to conceive of learning as a general process or mechanism that evolved early and that has been conserved throughout subsequent evolutionary adaptation. By contrast, ethologists have generally favored the stance that learning reflects niche-specific adaptation to a multitude of problems of survival. Animals did not evolve a single, stand-alone learning process, but rather domain- and task-specific learning embedded within individual adaptive behavioral systems, so there should be a multiplicity of learning rules, each tailored to a particular task or problem.

There is no single resolution of these conflicting views. In some sense both are nearly self-evident. After all, most animals are faced with the same basic problems of space and time, and most possess the same cellular and molecular machinery that make plasticity possible, so some basic laws of association might be quite general (e.g., Anokhin, 1974). Despite early claims for the demise of the idea of general learning processes (Seligman, 1970), it is fair to say that many basic learning phenomena are observed across a wide range of behavioral systems, both in the laboratory and in the field, often applying even in cases of relatively specialized learning. Likewise, at some level, learned behavior reflects the action of behavioral systems that evolved because they served an important function in the natural history of the species. Appreciation for the constraints on those systems in the animal’s natural environment is indispensable for understanding learning in more contrived, but often more tractable, laboratory situations.

Despite the central role played by evolution in ethology, we have said very little about questions of the evolution of learning and behavior.Acomprehensive theory about the evolution of learning remains elusive. Valuable contributions to an evolutionary theory of learning have been made in recent decades (e.g., Alexander, 1990; Tierney, 1986). However, one of the major challenges that remains a roadblock to a comprehensive theory is that we still do not understand well the costs and benefits of learning in terms of reproductive fitness (Johnston, 1981,1982).What is clear, as discussed earlier, is that one cannot assume that organizing the development or maintenance of a behavioral trait based on learning will always be beneficial. Mistakes can be made in any system requiring learning, which can have potentially disastrous consequences. What is more difficult to measure is the possibility that learning systems require a more complex and therefore costly brain or genome (Johnston, 1982). Speculations of this sort have been around for years, but there is still no rigorous metric by which to ascertain these potential costs.

Although a true synthesis of the approaches of experimental psychology and ethology may still elude us, the two fields have had mutually beneficial influences on one another. The importance of ecological validity is now generally recognized by experimental psychologists investigating learning processes in animals. Similarly, the value of a detailed experimental analysis of a learning process in a relatively artificial situation is now clearly appreciated by ethologists and behavioral ecologists. Finally, the shared methods, questions, and orientations of neuroscience increasingly pervade the study of both ethology and experimental psychology. This multidisciplinary synergy bodes well for the future of the study of animal learning.

Bibliography:

1. Adret, P. (1993). Vocal learning induced with operant techniques: An overview. Netherlands Journal of Zoology, 43, 125–142.
2. Akins, C. K., Domjan, M., & Gutierrez, G. (1994). Topography of sexually conditioned behavior in male Japanese-quail (coturnixjaponica) depends on the CS-US interval. Journal of Experimental Psychology: Animal Behavior Processes, 20, 199–209.
3. Alexander, R. D. (1990). Epigenetic rules and Darwinian algorithms: The adaptive study of learning and development. Ethology and Sociobiology, 11, 241–303.
4. Alkon, D. L. (1983). Learning in a marine snail. Scientific American, 249, 64–74.
5. Anokhin, P. K. (1974). Biology and neurophysiology of the conditional reflex and its role in adaptive behavior. New York: Pergamon Press.
6. Appeltants, D., Absil, P., Balthazart, J., & Ball, G. F. (2000). Identification of the origin of catecholaminergic inputs to HVc in canaries by retrograde tract tracing combined with tyrosine hydroxylase immunocytochemistry. Journal of Chemical Neurochemistry, 18, 117–133.
7. Ball, G. F. (1994). Neurochemical specializations associated with vocal learning and production in songbirds and budgerigars. Brain, Behavior, and Evolution, 44, 234–246.
8. Ball, G. F., & Hulse, S. H. (1998). Bird song. American Psychologist, 53, 37–58.
9. Balthazart, J. (1989). Steroid metabolism and the activation of social behavior. In J. Balthazart (Ed.), Advances in comparative and environmental physiology (Vol. 3, pp. 105–159). Berlin: Springer-Verlag.
10. Balthazart, J., Absil, P., Gérard, M., Appeltants, D., & Ball, G. F. (1998). Appetitive and consummatory male sexual behavior in Japanese quail are differentially regulated by subregions of the preoptic medial nucleus. Journal of Neuroscience, 18, 6512– 6527.
11. Balthazart, J., & Ball, G. F. (1995). Sexual differentiation of brain and behavior in birds. Trends in Endocrinology and Metabolism, 6, 21–29.
12. Balthazart, J., & Ball, G. F. (1998). The Japanese quail as a model systemfortheinvestigationofsteroid-catecholamineinteractions mediating appetitive and consummatory aspects of male sexual behavior. Annual Review of Sex Research, 9, 96–176.
13. Balthazart, J., De Clerck, A., & Foidart, A. (1992). Behavioral demasculinization of female quail is induced by estrogens: Studies with the new aromatase inhibitor, R76713. Hormones and Behavior, 26, 179–203.
14. Balthazart, J., Reid, J.,Absil, P., Foidart,A., & Ball, G. F. (1995).Appetitive as well as consummatory aspects of male sexual behavior in quail are activated by androgens and estrogens. Behavioral Neuroscience, 109, 485–501.
15. Balthazart, J., & Surlemont, C. (1990). Androgen and estrogen action in the preoptic area and activation of copulatory behavior in quail. Physiology and Behavior, 48, 599–609.
16. Balthazart, J., Surlemont, C., & Harada, N. (1992). Aromatase as a cellular marker of testosterone action in the preoptic area. Physiology and Behavior, 51, 395–409.
17. Baptista, L. F. (1996). Nature and nurturing in avian vocal development. In D. E. Kroodsma & E. H. Miller (Eds.), Ecology and evolution of acoustic communication in birds (pp. 39–60). Ithaca, NY: Cornell University Press.
18. Baptista, L. F., & Gaunt, S. L. L. (1997). Social interactions and vocal development in birds. In C. T. Snowdon & M. Hausberger (Eds.), Social influences on vocal development (pp. 23–40). Cambridge, UK: Cambridge University Press.
19. Baptista, L. F., & Petrinovich, L. (1984). Social interactions, sensitive phases and the song template hypothesis in the whitecrowned sparrow. Animal Behaviour, 32, 172–181.
20. Baptista, L. F., & Schuchmann, K. L. (1990). Song learning in the anna hummingbird. Ethology, 84, 15–26.
21. Barker, L. M., & Smith, J. C. (1974). A comparison of taste aversions induced by radiation and lithium chloride in CS-US and US-CS paradigms. Journal of Comparative and Physiological Psychology, 87, 644–654.
22. Bateson, P. P. G. (1966). The characteristics and context of imprinting. Biological Reviews, 41, 177–220.
23. Baum, M. J. (1995). Reassessing the role of medial preoptic area/ anterior hypothalamic neurons in appetitive aspects of masculine sexualbehavior.InJ.Bancroft(Ed.),Thepharmacologyofsexual function and dysfunction (pp. 133–139). Amsterdam: Elsevier Science.
24. Baxter, M. G., Bucci, D. J., Holland, P. C., & Gallagher, M. (1999). Impairments in conditioned stimulus processing after combined selective removal of hippocampal and cortical cholinergic input. Behavioral Neuroscience, 113, 486–495.
25. Baxter, M. G., Gallagher, M., & Holland, P. C. (1999). Blocking can occur without losses in attention in rats with selective removal of hippocampal cholinergic input. Behavioral Neuroscience, 113, 881–890.
26. Baxter, M. G., Holland, P. C., & Gallagher, M. (1997). Disruptionof attentional processing by selective removal of hippocampal cholinergic input. Journal of Neuroscience, 17, 5230–5236.
27. Baxter, M. G., Parker, A., Lindner, C. C. C., Izquierdo, A. D., & Murray, E. A. (2000). Control of response selection by reinforcer value requires interaction of amygdala and orbital prefrontal cortex. Journal of Neuroscience, 20, 4311–4319.
28. Beer, C. G. (1963). Ethology: The zoologists’approach to behavior. Tuatara, 11, 170–177.
29. Beer, C. G. (1975). Was professor Lehrman an ethologist? Animal Behaviour, 23, 957–964.
30. Benton, S., Nelson, D. A., Marler, P., & DeVoogd, T. J. (1998). Anterior forebrain pathway is needed for stable song expression in adult male white-crowned sparrows (Zonotrichia leucophrys). Behavioral Brain Research, 96, 135–150.
31. Bliss, T. V. P., & Collingridge, G. L. (1993). A synaptic model of memory: Long-term potentation in the hippocampus. Nature, 361, 31–39.
32. Boice, R., & Denny, M. R. (1965). The conditioned licking response in rats as a function of the CS-UCS interval. Psychonomic Science, 3, 93–94.
33. Bolhuis, J. J. (1991). Mechanisms of avian imprinting: A review. Biological Reviews of the Cambridge Philosophical Society, 66, 303–345.
34. Bonardi, C. (1998). Conditional learning: An associative analysis. In N. A. Schmajuk & P. C. Holland (Eds.), Occasion setting: Associative learning and cognition in animals (pp. 37–67). Washington, DC: American Psychological Association.
35. Bottjer, S. W., Halsema, K. A., Brown, S. A., & Miesner, E. A. (1989). Axonal connections of a forebrain nucleus involved with vocal learning in zebra finches. Journal of Comparative Neurology, 279, 312–326.
36. Bottjer, S. W., Miesner, E. A., & Arnold, A. P. (1984). Forebrain lesions disrupt development but not maintenance of song in passerine birds. Science, 224, 901–903.
37. Boysson-Bardies, B. D., & Vihman, M. M. (1991). Adaptation to language: Evidence from babbling and first words in four languages. Language, 67, 297–319.
38. Brainard, M. S., & Doupe, A. J. (2000). Auditory feedback in learning and maintenance of vocal behaviour. Nature Reviews Neuroscience, 1, 31–40.
39. Brandon, S. E., & Wagner, A. R. (1998). Occasion setting: Influences of conditional emotional responses and configural cues. In N. A. Schmajuk & P. C. Holland (Eds.), Occasion setting: Associative learning and cognition in animals (pp. 343–382). Washington, DC: American Psychological Association.
40. Brenowitz, E. A. (1997). Comparative approaches to the avian song system. Journal of Neurobiology, 33, 517–531.
41. Brenowitz, E. A., Margoliash, D., & Nordeen, K. W. (1997). An introduction to birdsong and the avian song system. Journal of Neurobiology, 33, 495–500.
42. Brown, T. H., Kairiss, E. W., & Keenan, C. L. (1990). Hebbian synapses: Biophysical mechanisms and algorithms. Annual Review of Neuroscience, 13, 475–511.
43. Bucci, D. J., Holland, P. C., & Gallagher, P. C. (1998). Removal of cholinergic input to rat posterior parietal cortex disrupts incremental processing of conditioned stimuli. Journal of Neuroscience, 18, 8038–8046.
44. Catchpole, C. K., & Slater, P. J. B. (1995). Bird song: Biological themes and variations. New York: Cambridge University Press.
45. Chaiken, M., Böhner, J., & Marler, P. (1993). Song acquisition in European starlings, Sturnus vulgaris: A comparison of the songs of live-tutored, tape-tutored, untutored, and wild-caught males. Animal Behaviour, 46, 1079–1090.
46. Chomsky, N. (1965). Aspects of a theory of syntax. Cambridge, MA: MIT Press.
47. Chomsky, N. (1980). Rules and representations. New York: Columbia University Press.
48. Clayton, D. F. (1997). Role of gene regulation in song circuit developmentandsonglearning.JournalofNeurobiology,33,549–571.
49. Clayton, N. S., & Soha, J. A. (1999). Memory in avian food caching and song learning: A general mechanism or different processes. Advances in the Study of Behavior, 28, 115–173.
50. Coldwell, S. E., & Tordoff, M. G. (1993). Learned preferences for the flavor of salted food. Physiology and Behavior, 54, 999– 1004.
51. Colwill, R. M., & Motzkin, D. K. (1994). Encoding of the unconditioned stimulus in Pavlovian conditioning. Animal Learning and Behavior, 22, 384–394.
52. Colwill, R. M., & Rescorla, R. A. (1985). Post-conditioning devaluation of a reinforcer affects instrumental responding. Journal of Experimental Psychology: Animal Behavior Processes, 11, 120– 132.
53. Constantine-Paton, M., Cline, H. T., & Debskie, E. (1990). Patterned activity, synaptic convergence and the NMDA receptor in developing visual pathways. Annual Review of Neuroscience, 13, 129–154.
54. Craig, W. (1918). Appetites and aversions as constituents of instincts. Biological Bulletin, 34, 91–107.
55. Crawford, L. L., Akins, C. K., & Domjan, M. (1994). Stimulus control of copulatory behavior in sexually naive male Japanese quail (coturnix japonica): Effects of test context and stimulus movement. Journal of Comparative Psychology, 108, 252–261.
56. Crawford, L. L., & Domjan, M. (1995). 2nd-order sexual conditioning in male Japanese quail (coturnix japonica). Animal Learning and Behavior, 23, 327–334.
57. Crawford, L. L., & Domjan, M. (1996). Conditioned inhibition of social approach in male Japanese quail (Coturnix japonica) using visualexposuretoafemale.BehaviouralProcesses,36,163–169.
58. Crawford, L. L., Holloway, K. S., & Domjan, M. (1993). The nature of sexual reinforcement. Journal of the Experimental Analysis of Behavior, 60, 55–66.
59. Cusato, B., & Domjan, M. (2000). Facilitation of appetitive conditioning with naturalistic conditioned stimuli: CS and US factors. Animal Learning and Behavior, 28, 247–256.
60. Dave, A. S., Yu, A. C., & Margoliash, D. (1998). Behavioral state modulation of auditory activity in a vocal motor system. Science, 282, 2250–2254.
61. Davidson, T. L. (1993). The nature and function of interoceptive signals to feed: Toward an integration of physiological and learning perspectives. Psychological Review, 100, 640–657.
62. Dickinson, A. (1980). Contemporary animal learning theory. London: Cambridge University Press.
63. Dickinson, A., & Mackintosh, N. J. (1978). Classical conditioning in animals. Annual Review of Psychology, 29, 587–612.
64. Dickinson, A., Smith, J., & Mirenowicz, J. (2000). Dissociation of Pavlovian and instrumental incentive learning under dopamine antagonists. Behavioral Neuroscience, 114, 468–483.
65. Domjan, M. (1987). Photoperiodic and endocrine control of social proximity behavior in male Japanese quail (Coturnix coturnix japonica). Behavioral Neuroscience, 101, 385–392.
66. Domjan, M. (1994). Formulation of a behavior system for sexual conditioning. Psychonomic Bulletin and Review, 4, 421–428.
67. Domjan, M., Akins, C., & Vandergriff, D. H. (1992). Increased responding to female stimuli as a result of sexual experience: Tests of mechanisms of learning. Quarterly Journal of Experimental Psychology, 45B, 139–157.
68. Domjan, M., Blesbois, E., & Williams, J. (1998). The adaptive significance of sexual conditioning: Pavlovian control of sperm release. Psychological Science, 9(5), 411–415.
69. Domjan, M., Greene, P., & North, N. C. (1989). Contextual conditioning and the control of copulatory behavior by speciesspecific stimuli in male Japanese quail. Journal of Experimental Psychology: Animal Behavior Processes, 15, 147–153.
70. Domjan, M., & Hall, S. (1986a). Determinants of social proximity in Japanese quail (Coturnix coturnix japonica): Male behavior. Journal of Comparative Psychology, 100, 59–67.
71. Domjan, M., & Hall, S. (1986b). Sexual dimorphism in the social proximity behavior of Japanese quail (Coturnix coturnix japonica). Journal of Comparative Psychology, 100, 68–71.
72. Domjan, M., & Hollis, K. L. (1988). Reproductive behavior: A potential model system for adaptive specializations in learning. In R. C. Bolles & M. D. beecher (Eds.), Evolution and learning (pp. 213–237). Hillsdale, NJ: Erlbaum.
73. Domjan, M., Lyons, R., North, N. C., & Bruell, J. (1986). Sexual Pavlovian conditioned approach behavior in male Japanese quail (Coturnix coturnix japonica). Journal of Comparative Psychology, 100, 413–421.
74. Domjan, M., O’Nary, D., & Greene, P. (1988). Conditioning of appetitive and consummatory sexual behavior in male Japanese quail. Journal of the Experimental Analysis of Behavior, 50, 505–519.
75. Domjan, M., & Wilson, N. E. (1972). Specificity of cue to consequence in aversion learning in the rat. Psychonomic Science, 26, 143–145.
76. Elman, J. L., Bates, E. A., Johnson, M. H., Karmiloff-Smith, A., Parisi, D., & Plunkett, K. (1996). Rethinking innateness. Cambridge, MA: MIT Press.
77. Everitt, B. J. (1995). Neuroendocrine mechanisms underlying appetitive and consummatory elements of masculine sexual behavior. In J. Bancroft (Ed.), The pharmacology of sexual function and dysfunction (pp. 15–31). Amsterdam: Elsevier.
78. Everitt, B. J., & Robbins, T. W. (1992). Amygdala-ventral striatal interactions and reward-related processes. In J. Aggleton (Ed.), The Amygdala: Neurological aspects of emotion, memory, and mental dysfunction (pp. 401–429). Chichester, UK: Wiley.
79. Fanselow, M. S. (1994). Neural organization of the defensive behavior system responsible for fear. Psychonomic Bulletin and Review, 4, 429–438.
80. Fanselow, M. S., & Lester, L. S. (1985). A functional behavioristic approach to aversively motivated behavior: Predatory imminence as a determinant of the topography of defensive behavior. In R. C. Bolles & M. D. Beecher (Eds.), Evolution and learning (pp. 185–212). Hillsdale, NJ: Erlbaum.
81. Farabaugh, S. M., & Dooling, R. J. (1996). Acoustic communication in parrots: Laboratory and field studies of budgerigars, Melopsittacus undulatus. In D. E. Kroodsma & E. H. Miller (Eds.), Ecology and evolution of acoustic communication in birds (pp. 97–118). Ithaca, NY: Cornell University Press.
82. Fodor, J. (1983). The modularity of mind. Cambridge, MA: MIT Press.
83. Foidart, A., & Balthazart, J. (1995). Sexual differentiation of brain and behavior in quail and zebra finches: Studies with a new aromatase inhibitor, R76713. Journal of Steroid and Biochemical Molecular Biology, 53, 267–275.
84. Fortune, E. S., & Margoliash, D. (1995). Parallel pathways and convergence onto HVc and adjacent neostriatum of adult zebra finches (Taeniopygia guttata). Journal of Comparative Neurology,360,413–441.
85. Gallagher, M., Graham, P. W., & Holland, P. C. (1990). The amygdala central nucleus and appetitive Pavlovian conditioning: Lesions impair one class of conditioned performance. Journal of Neuroscience, 10, 1906–1911.
86. Gallagher, M., & Holland, P. (1992). Understanding the function of the central nucleus: Is simple conditioning enough? In J. P. Aggleton (Ed.), The amygdala: Neurobiological aspects of emotion, memory, and mental dysfunction (pp. 307–321). New York: Wiley-Liss.
87. Gallagher, M., McMahan, R. W., & Schoenbaum, G. (1999). Orbitofrontal cortex and representation of incentive value in associative learning. Journal of Neuroscience, 19, 6610–6614.
88. Gallistel, C. R., & Gibbon, J. (2000). Time, rate, and conditioning. Psychological Review, 107, 289–344.
89. Garcia, J., Ervin, F. R., & Koelling, R.A. (1966). Learning with prolongeddelayofreinforcement.PsychonomicScience,5,121–122.
90. Garcia, J., & Koelling, R. A. (1966). Relation of cue to consequence in avoidance learning. Psychonomic Science, 4, 123–124.
91. Garcia, J., Lasiter, P. S., Bermudez-Rattoni, F., & Deems, D. A. (1985). A general theory of aversion learning. Annals of the New York Academy of Sciences, 443, 8–21.
92. Gibbon, J. (1977). Scalar expectancy theory and Weber’s law in animal timing. Psychological Review, 84, 279–325.
93. Gibbon, J., Baldock, M. D., Locurto, C. M., Gold, L., & Terrace, H. S. (1977). Trial and intertrial durations in autoshaping. Journal of Experimental Psychology: Animal Behavior Processes, 3, 264–284.
94. Gibbon, J., & Church, R. M. (1990). Representation of time. Cognition, 37, 23–54.
95. Gormezano, I., & Moore, J. W. (1969). Classical conditioning. In M. H. Marx (Ed.), Learning: Processes (pp. 121–203). London: MacMillan.
96. Gould, J. L. (1982). Ethology: The mechanisms and evolution of behavior. New York: W. W. Norton.
97. Hall, G. (1996). Learning about associatively-activated stimulus representations: Implications for acquired equivalence and perceptual learning. Animal Learning and Behavior, 24, 233–255.
98. Hall,G.,&Mondragon,E.(1998).Contextualcontrolasoccasionsetting. In N.A. Schmajuk & P. C. Holland (Eds.), Occasion setting: Associative learning and cognition in animals (pp. 199–222). Washington, DC:American PsychologicalAssociation.
99. Hall, J., Parkinson, J. A., Connor, T. M., Dickinson, A., & Everitt, B. J. (2001). Involvement of the central nucleus of the amygdala and nucleus accumbens core in mediating Pavlovian influences on instrumental behavior. European Journal of Neuroscience, 13, 1984–1992.
100. Han, J.-S., Gallagher, M., & Holland, P. C. (1995). Hippocampal lesions disrupt decrements but not increments in conditioned stimulus processing. Journal of Neuroscience, 11, 7323–7329.
101. Han, J.-S., Holland, P. C., & Gallagher, M. (1999). Disconnection of amygdala central nucleus and substantia innominata/nucleus basalis disrupts increments in conditioned stimulus processing. Behavioral Neuroscience, 113, 143–151.
102. Han, J.-S., McMahan, R. W., Holland, P. C., & Gallagher, M. (1997). The role of an amygdalo-nigrostriatal pathway in associative learning. Journal of Neuroscience, 17, 3913–3919.
103. Hatfield, T., Han, J.-S., Conley, M., Gallagher, M., & Holland, P. (1996). Neurotoxic lesions of the basolateral, but not central, amygdala interfere with Pavlovian second-order conditioning and reinforcer-devaluation effects. Journal of Neuroscience, 16, 5256–5265.
104. Hawkins, R. D., & Kandel, E. R. (1984). Is there a cell-biological alphabet for simple forms of learning? Psychological Review, 91, 375–391.
105. Hess, E. H. (1973). New York: Van Nostrand.
106. Higa, J. J. (1998). Interval timing: Is there a clock? Behavioural Processes, 44, 87–88.
107. Hilgard, E.R.,&Bower,G.H.(1975).Theories of learning(4th ed.). Englewood Cliffs, NJ: Prentice Hall.
108. Hilliard, S., & Domjan, M. (1995). Effects on sexual conditioning of devaluing the US through satiation. Quarterly Journal of Experimental Psychology, 48B, 84–92.
109. Hinde, R. A. (1953). Appetitive behaviour, consummatory act, and the hierarchical organization of behaviour—with special reference to the great tit (Parus major). Behaviour, 5, 189–224.
110. Hinde, R. A. (1960). Energy models of motivation. Symposia of the Society of Experimental Biology, 14, 199–213.
111. Hinde, R. A. (1970). Animal behaviour (2nd ed.). New York: McGraw-Hill.
112. Hinde, R. A. (1982). Ethology: Its nature and relations with other sciences. New York: Oxford University Press.
113. Hodos, W., & Campbell, C. B. G. (1969). Scala Naturae: Why there is no theory in comparative psychology. Psychological Review, 76, 337–350.
114. Hoffman, H. S., & Ratner, A. M. (1973). A reinforcement model of imprinting: Implications for socialization in monkeys and men. Psychological Review, 80, 527–544.
115. Holland, P. C. (1977). Conditioned stimulus as a determinant of the form of the Pavlovian conditioned response. Journal of Experimental Psychology: Animal Behavior Processes, 3, 77–104.
116. Holland,P.C.(1979).Theeffectsofqualitativeandquantitativevariation in the US on individual components of Pavlovian appetitive conditioned behavior in rats. Animal Learning and Behavior, 7, 424–432.
117. Holland, P. C. (1980a). CS-US interval as a determinant of the form of Pavlovian appetitive conditioned responses. Journal of Experimental Psychology: Animal Behavior Processes, 6, 155–174.
118. Holland, P. C. (1980b). Influence of visual conditioned stimulus characteristics on the form of Pavlovian appetitive conditioned responding in rats. Journal of Experimental Psychology: Animal Behavior Processes, 6, 81–97.
119. Holland, P. C. (1981a). Acquisition of representation-mediated conditioned food aversions. Learning and Motivation, 12, 1–18.
120. Holland, P. C. (1981b). The effects of satiation after first- and second-order appetitive conditioning. Pavlovian Journal of Biological Science, 16, 18–24.
121. Holland, P. C. (1983a). Occasion-setting in Pavlovian feature positive discriminations. In M.L. Commons, R. J. Herrnstein, &A.R. Wagner (Eds.), Quantitative analyses of behavior: Discrimination processes (Vol. 4, pp. 183–206). NewYork: Ballinger.
122. Holland, P. C. (1983b). Representation-mediated overshadowing and potentiation of conditioned aversions. Journal of Experimental Psychology: Animal Behavior Processes, 9, 1–13.
123. Holland, P. C. (1984a). Differential effects of reinforcement of an inhibitory feature after serial and simultaneous feature negative discrimination training. Journal of Experimental Psychology: Animal Behavior Processes, 10, 461–475.
124. Holland, P. C. (1984b). The origins of Pavlovian conditioned behavior. In G. Bower (Ed.), The psychology of learning and motivation (Vol. 18, pp. 129–173). Englewood Cliffs, NJ: Prentice Hall.
125. Holland, P. C. (1985). Element pretraining influences the content of appetitive serial compound conditioning in rats. Journal of Experimental Psychology: Animal Behavior Processes, 11, 367–387.
126. Holland, P. C. (1988). Excitation and inhibition in unblocking. Journal of Experimental Psychology: Animal Behavior Processes, 14, 261–279.
127. Holland, P. C. (1990a). Event representation in Pavlovian conditioning: Image and action. Cognition, 37, 105–131.
128. Holland, P. C. (1990b). Forms of memory in Pavlovian conditioning. In J. L. McGaugh, N. M. Weinberger, & G. Lynch, (Eds.), Brain organization and memory: Cells, systems, and circuits (pp. 78–105). New York: Oxford University Press.
129. Holland, P. C. (1991). Learning, thirst, and drinking. In D. A. Booth & D. J. Ramsay (Eds.), Thirst: Physiological and psychological aspects (pp. 278–294). Berlin: Springer-Verlag.
130. Holland, P. C. (1992). Occasion setting in Pavlovian conditioning. In D. Medin (Ed.), The psychology of learning and motivation (Vol. 28, pp. 69–125). San Diego, CA: Academic Press.
131. Holland, P. C. (1999). Overshadowing and blocking as acquisition deficits: No recovery after extinction of overshadowing or blocking cues. Quarterly Journal of Experimental Psychology, 52B, 307–333.
132. Holland, P. C. (2000). Trial and intertrial durations in appetitive conditioning in rats. Animal Learning and Behavior, 28, 121–135.
133. Holland, P. C., & Block, H. (1983). Evidence for a unique cue in positive patterning. Bulletin of the Psychonomic Society, 21, 297– 300.
134. Holland, P. C., & Bouton, M. E. (1999). Hippocampus and context in classical conditioning. Current Opinion in Neurobiology, 9, 195– 202.
135. Holland,P.C.,Chik,Y.,&Zhang,Q.(2001).Inhibitorylearningtests of conditioned stimulus associability in rats with lesions of the amygdala central nucleus. Behavioral Neuroscience, 115, 1154– 1158.
136. Holland, P. C., & Forbes, D. T. (1982a). Control of conditional discrimination performance by CS-evoked event representations. Animal Learning and Behavior, 10, 249–256.
137. Holland, P. C., & Forbes, D. T. (1982b). Representation-mediated extinction of flavor aversions. Learning and Motivation, 13, 454– 471.
138. Holland, P. C., & Gallagher, M. (1993a). Amygdala central nucleus lesions disrupt increments, but not decrements, in CS processing. Behavioral Neuroscience, 107, 246–253.
139. Holland, P. C., & Gallagher, M. (1993b). Effects of amygdala central nucleus lesions on blocking and unblocking. Behavioral Neuroscience, 107, 235–245.
140. Holland, P. C., & Gallagher, M. (1999). Amygdala circuitry in attentional and representational processes. Trends in Cognitive Sciences, 3, 65–73.
141. Holland, P. C., Hatfield, T., & Gallagher, M. (2001). Rats with lesions of basolateral amygdala show normal increases in conditioned stimulus processing but reduced conditioned potentiation of eating. Behavioral Neuroscience, 115, 945–950.
142. Holland,P.C.,&Rescorla,R.A.(1975).Theeffectoftwowaysofdevaluing the unconditioned stimulus after first- and second-order appetitive conditioning. Journal of Experimental Psychology: Animal Behavior Processes, 1, 355–363.
143. Holland, P. C., & Ross, R. T. (1981). Within-compound associations in serial compound conditioning. Journal of Experimental Psychology: Animal Behavior Processes, 7, 228–241.
144. Holland, P. C., & Straub, J. J. (1979). Differential effects of two ways of devaluing the unconditioned stimulus after Pavlovian appetitive conditioning. Journal of Experimental Psychology: Animal Behavior Processes, 5, 65–78.
145. Holland, P. C., Thornton, J. A., & Ciali, L. (2000). The influence of associability changes in negative patterning and other discriminations. Journal of Experimental Psychology: Animal Behavior Processes, 26, 462–476.
146. Holloway, K. S., & Domjan, M. (1993). Sexual approach conditioning: Tests of unconditioned stimulus devaluation using hormone manipulations. Journal of Experimental Psychology: Animal Behavior Processes, 19, 47–55.
147. Hull, C. L. (1943). Principles of behavior. New York: AppletonCentury-Crofts.
148. Hull, E. M. (1995). Dopaminergic influences on male rat sexual behavior. In P. E. Micevych & R. P. J. Hammer (Eds.), Neurobiological effects of sex steroid hormones (pp. 234–253). Cambridge, UK: Cambridge University Press.
149. Hull, E. M., Du, J. F., Lorrain, D. S., & Matuszewich, L. (1997). Testosterone, preoptic dopamine, and copulation in male rats. Brain Research Bulletin, 44, 327–333.
150. Hultsch, H., & Todt, D. (1989). Memorization and reproduction of songs in nightingales (Luscinia megarynchos): Evidence for package formation. Journal of Comparative Physiology, 165A, 197–203.
151. Immelmann, K., & Beer, C. G. (1989). A dictionary of ethology. Cambridge, MA: Harvard University Press.
152. Jarvis, E. D., Scharff, C., Grossman, M. R., Ramos, J. A., & Nottebohm, F. (1998). For whom the bird sings: contextdependent gene expression. Neuron, 21, 775–788.
153. Johnston, T. D. (1981). Contrasting approaches to a theory of learning. Behavioral and Brain Sciences, 4, 125–138.
154. Johnston, T. D. (1982). Selective costs and benefits in the evolution of learning. Advances in the Study of Behavior, 12, 65–106.
155. Jusczyk,P.W.(1997).Cambridge, MA: MIT Press.
156. Kacelnik, A., & Bateson, M. (1997). Risk sensitivity: Cross-roads for theories of decision making. Trends in Cognitive Sciences, 1, 304–309.
157. Kacelink, A., & Krebs, J. R. (1997). Yanomamo dreams and starling payloads: The logic of optimality. In L. Betzig (Ed.), Human nature (pp. 21–35). New York: Oxford University Press.
158. Kamin, L. J. (1968). Attention-like processes in classical conditioning. In M. R. Jones (Ed.), Miami Symposium on the Prediction of Behavior: Aversive Stimulation (pp. 9–32). Coral Gables, FL: University of Miami Press.
159. Kandel, E. R. (1976). Cellular basis of behavior. San Francisco: W. H. Freeman.
160. Kelley, D. B., & Nottebohm, F. (1979). Projections of a telencephalic auditory nucleus-field L-in the canary. Journal of Comparative Neurology, 183, 455–470.
161. Kendler, H. (1946). The influence of simultaneous hunger and thirst drives upon the learning of two opposed spatial responses of the white rat. Journal of Experimental Psychology, 36, 212– 220.
162. Koksal, F., Domjan, M., & Weisman, G. (1994). Blocking of the sexual conditioning of differentially effective conditioned stimulus objects. Animal Learning and Behavior, 22, 103–111.
163. Konishi, M. (1965). The role of auditory feedback in the control of vocalization in the white-crowned sparrow. Zeitschrift für Tierpsychologie, 22, 770–783.
164. Konishi, M. (1985). Birdsong: From behavior to neuron. Annual Review of Neuroscience, 8, 125–170.
165. Konishi, M. (1989). Birdsong for neurobiologists. Neuron, 3, 541– 549.
166. Krane, R. V., & Wagner, A. R. (1975). Taste aversion learning with a delayed shock US: Implications for the “generality of the laws of learning.” Journal of Comparative Physiological Psychology, 88, 882–889.
167. Kremer, E. (1978). The Rescorla-Wagner model: Losses in associative strength in compound conditioned stimuli. Journal of Experimental Psychology: Animal Behavior Processes, 4, 22–36.
168. Kroodsma, D. E. (1982). Learning and the ontogeny of sound signals in birds. In D. E. Kroodsma & E. H. Miller (Eds.), Acoustic communication in birds (Vol. 2, pp. 1–23). New York: Academic Press.
169. Kroodsma, D. E. (1988). Contrasting styles of song development and their consequences among passerine birds. In R. C. Bolles & M. D. Beecher (Eds.), Evolution and learning (pp. 157–184). Hillsdale, NJ: Erlbaum.
170. Kroodsma, D. E. (1996). Ecology of passerine song development. In D. E. Kroodsma & E. H. Miller (Eds.), Ecology and evolution of acoustic communication in birds (pp. 3–19). Ithaca, NY: Cornell University Press.
171. Kroodsma, D. E., & Baylis, J. R. (1982). Appendix: A world survey of evidence for vocal learning in birds. In D. E. Kroodsma & E. H. Miller (Eds.), Acoustic communication in birds (Vol. 2, pp. 311–337). New York: Academic Press.
172. Kroodsma, D., & Konishi, M. (1991). A suboscine bird (Eastern phoebe, Sayornis phoebe) develops normal song without auditory feedback. Animal Behaviour, 42, 477–487.
173. Lack, D. (1939). The behaviour of the robin: Pts. 1 and 2. Proceedings of the Zoological Society of London, 109, 169–178.
174. Lehrman, D. S. (1953). A critique of Konrad Lorenz’s theory of instinctive behavior. Quarterly Review of Biology, 28, 337–363.
175. Lenneberg, E. (1967). Biological foundations of language. New York: Wiley.
176. Liu, Y. C., Salamone, J. D., & Sachs, B. D. (1997). Lesions in medial preoptic area and bed nucleus of stria terminalis: Differential effects on copulatory behavior and noncontact erection in male rats. Journal of Neuroscience, 17, 5245–5253.
177. Lorenz, K. (1950). The comparative method in studying innate behavior patterns. Symposia of the Society for Experimental Biology, 4, 221–268.
178. Lorenz, K. (1965). Evolution and modification of behavior. Chicago: University of Chicago Press.
179. Lorenz, K. (1981). The foundations of ethology. New York: Simon and Schuster.
180. Lovejoy, A. O. (1936). The great chain of being. Cambridge, MA: Harvard University Press.
181. Mackintosh, N. J. (1974). The psychology of animal learning. New York: Academic Press.
182. Mackintosh, N. J. (1975). A theory of attention: Variations in the associability of stimuli with reinforcement. Psychological Review, 82, 276–298.
183. Malkova, L., Gaffan, D., & Murray, E. A. (1997). Excitotoxic lesions of the amygdala fail to produce impairment in visual learning for auditory secondary reinforcement but interfere with reinforcer devaluation effects in rhesus monkeys. Journal of Neuroscience, 17, 6011–6020.
184. Manabe, K., & Dooling, R. J. (1997). Control of vocal production in budgerigars (Melopsittacus undulatus): Selective reinforcement, call differentiation, and stimulus control. Behavioural Processes, 41, 117–132.
185. Margoliash, D. (1997). Functional organization of forebrain pathways for song production and perception. Journal of Neurobiology, 33, 671–693.
186. Marler, P. R. (1952). Variations in the song of the chaffinch (Fingilla coelebs). Ibis, 94, 458–472.
187. Marler, P. R. (1961). The filtering of external stimuli during instinctive behavior. In W. H. Thorpe & O. L. Zangwill (Eds.), Current problems in animal behavior (pp. 150–166). Cambridge, UK: Cambridge University Press.
188. Marler, P. R. (1970). A comparative approach to vocal learning: Song development in white-crowned sparrows. Journal of Comparative and Physiological Psychology, 71(Suppl.), 1–25.
189. Marler, P. R. (1987). Sensitive periods and the roles of specific and general sensory stimulation in birdsong learning. In J. P. Rauschecker & P. Marler (Eds.), Imprinting and cortical plasticity (pp. 99–135). New York: Wiley.
190. Marler, P. R. (1991). Song-learning: The interface with neuroethology. Trends in Neuroscience, 14, 199–206.
191. Marler, P. R. (1997). Three models of song learning: Evidence from behavior. Journal of Neurobiology, 33, 501–516.
192. Marler, P. R., & Hamilton, W. J. I. (1966). Mechanisms of animal behavior. New York: Wiley.
193. Marler, P. R., & Nelson, D. A. (1992). Neuroselection and song learning in birds: Species universals in a culturally transmitted behavior. Seminars in the Neurosciences, 4, 415–423.
194. Marler, P. R., & Nelson, D. A. (1993). Action-based learning: A new form of developmental plasticity in bird song. Netherlands Journal of Zoology, 43, 91–103.
195. Marler, P. R., & Peters, S. (1982a). Developmental overproduction and selective attrition: New processes in the epigenesis of birdsong. Developmental Psychobiology, 15, 269–378.
196. Marler, P. R., & Peters, S. (1982b). Subsong and plastic song: Their role in the vocal learning process. In D. E. Kroodsma & E. H. Miller (Eds.), Acoustic communication in birds (Vol. 2, pp. 25– 50). NewYork:Academic Press.
197. Marler, P. R., & Sherman, V. (1985). Innate differences in singing behavior of sparrows reared in isolation from adult conspecific song. Animal Behaviour, 33, 57–71.
198. Marler, P. R., & Tamura, M. (1964). Song “dialects” in three populations of white-crowned sparrows. Science, 146, 1483–1486.
199. Mayr, E. (1963). Animal species and evolution. Cambridge, MA: Belknap Press.
200. McCasland, J. S. (1987). Neuronal control of bird song production. Journal of Neuroscience, 7, 23–39.
201. Meck, W. H., & Church, R. M. (1982). Abstraction of temporal attributes. Journal of Experimental Psychology: Animal Behavior Processes, 8, 226–243.
202. Meddle, S. L., King, V. M., Follett, B. K., Wingfield, J. C., Ramenofsky, M., Foidart, A., & Balthazart, J. (1997). Copulation activates Fos-like immunoreactivity in the male quail forebrain. Behavioral Brain Research, 85, 143–159.
203. Meisel, R. L., & Sachs, B. D. (1994). The physiology of male sexual behavior. In E. Knobil & J. D. Neill (Eds.), The physiology of reproduction (2nd ed., Vol. 2, pp. 3–105). NewYork: Raven Press.
204. Mello, C., Vicario, D. S., & Clayton, D. F. (1992). Song presentation induces gene expression in the songbird forebrain. Proceedings of the National Academy of Sciences, USA, 89, 6818–6822.
205. Miller, R. R., & Barnet, R. C. (1993). The role of time in elementary associations. Current Directions in Psychological Science, 2, 106– 111.
206. Miller, R. R., Barnet, R. C., & Grahame, N. J. (1995). Assessment of the Rescorla-Wagner Model. Psychological Bulletin, 117, 363–386.
207. Miller, R. R., McKinzie, D. L., Kraebel, K. S., & Spear, N. E. (1996). Changes in the expression of stimulus selection: Blocking represents selective memory retrieval rather than selective associations. Learning and Motivation, 27, 307–316.
208. Moss, C. F., & Shettleworth, S. J. (1996). Neuroethological studies of cognitive and perceptual processes. Boulder, CO: Westview Press.
209. Mowrer, O. H. (1947). On the dual nature of learning—a reinterpretation of conditioning and problem solving. Harvard Educational Review, 17, 102–148.
210. Myers, K. P., & Hall, W. G. (2000). Conditioned changes in appetitive and consummatory responses to flavors paired with oral or nutrient reinforcers among adult rats. Physiology and Behavior, 68, 603–610.
211. Nash, S., & Domjan, M. (1991). Learning to discriminate the sex of conspecifics in male japanese quail (Coturnix coturnix japonica): Tests of “biological constraints.” Journal of Experimental Psychology: Animal Behavior Processes, 17, 342–353.
212. Nash, S., Domjan, M., & Askins, M. (1989). Sexual-discrimination learning in male Japanese quail (Coturnix coturnix japonica). Journal of Comparative Psychology, 103, 347–358.
213. Nelson, D. A. (1992). Song overproduction and selective attrition lead to song sharing in the field sparrow (Spizella pusilla). Behavioral Ecology and Sociobiology, 30, 415–424.
214. Nelson, D. A. (1997). Social interactions and sensitive phases for song learning: A critical review. In C. T. Snowdon & M. Hausberger (Eds.), Social influences on vocal development (pp. 7–22). Cambridge, UK: Cambridge University Press.
215. Newport, E. (1990). Maturational constraints on language learning. Cognitive Science, 14, 11–28.
216. Nordeen, K. W., & Nordeen, E. J. (1992). Auditory feedback is necessary for the maintenance of stereotyped song in adult zebra finches. Behavioral and Neural Biology, 57, 58–66.
217. Nordeen, K. W., & Nordeen, E. J. (1997). Anatomical and synaptic substrates for avian song learning. Journal of Neurobiology, 33, 532–548.
218. Nottebohm, F. (1980). Brain pathways for vocal learning in birds: A review of the first 10 years. In J. M. Sprague & A. N. Epstein (Eds.), Progress in psychobiology and physiological psychology (Vol. 9, pp. 85–214). New York: Academic Press.
219. Nottebohm, F. (1993). The search for neural mechanisms that define the sensitive period for song learning in birds. Netherlands Journal of Zoology, 43, 193–234.
220. Nottebohm, F. (1996). The King Solomon lectures in neuroethology: A white canary on Mount Acropolis. Journal of Comparative Physiology, 179A, 149–156.
221. Nottebohm, F., Alvarez-Buylla, A., Cynx, J., Kirn, J., Ling, C. Y., Nottebohm, M., Suter, R., Tolles, A., & Williams, H. (1990). Song learning in birds: The relation between perception and production. Philosophical Transactions of the Royal Society of London, 329B, 115–124.
222. Nottebohm, F., Kelley, D. B., & Paton, J. A. (1982). Connections of vocal control nuclei in the canary telencephalon. Journal of Comparative Neurology, 207, 344–357.
223. Nottebohm, F., Stokes, T. M., & Leonard, C. M. (1976). Central control of song in the canary, Serinus canarius. Journal of Comparative Neurology, 165, 457–486.
224. Nowicki, S., & Marler, P. R. (1988). How do birds sing? Music Perception, 5, 391–426.
225. Okuhata, S., & Saito, N. (1987). Synaptic connections of thalamocerebral vocal control nuclei of the canary. Brain Research Bulletin, 18, 35–44.
226. Panzica, G. C., Viglietti-Panzica, C., & Balthazart, J. (1996). The sexually dimorphic medial preoptic nucleus of quail: Akey brain area mediating steroid action on male sexual behavior. Frontiers in Neuroendocrinology, 17, 51–125.
227. Pavlov, I. P. (1927). Conditioned reflexes: An investigation of the physiological activity of the cerebral cortex (G. V. Anrep, Trans.). London: Oxford University Press.
228. Pearce, J. M. (1987). A model for stimulus generalization in Pavlovian conditioning. Psychological Review, 94, 61–73.
229. Pearce, J. M. (1994). Similarity and discrimination: A selective review and a connectionist model. Psychological Review, 101, 587–607.
230. Pearce, J. M., Aydin, A., & Redhead, F. S. (1997). Configural analysis of summation in autoshaping. Journal of Experimental Psychology: Animal Behavior Processes, 23, 84–94.
231. Pearce, J. M., & Hall, G. (1980). A model for Pavlovian learning: Variations in the effectiveness of conditioned but not of unconditioned stimuli. Psychological Review, 106, 532–552.
232. Peters, S., Marler, P., & Nowicki, S. (1992). Song sparrows learn from limited exposure to song models. Condor, 94, 1016–1019.
233. Petrovich, G. D., Setlow, B., Holland, P. C., & Gallagher, M. (2001). Amygdalo-hypothalamic circuit is necessary for CS potentiation of feeding. Society for Neuroscience Abstract, 422,
234. Pinker, S. (1994). The language instinct. New York: William Morrow.
235. Rescorla, R. A. (1967). Pavlovian conditioning and its proper control procedures. Psychological Review, 74, 71–80.
236. Rescorla, R. A. (1968). Probability of shock in the presence and absence of CS in fear conditioning. Journal of Comparative and Physiological Psychology, 66, 1–5.
237. Rescorla,R.A.(1969a).Conditionedinhibitionoffearresultingfrom negative CS-US contingencies. Journal of Comparative and Physiological Psychology, 67, 504–509.
238. Rescorla, R. A. (1969b). Pavlovian conditioned inhibition. Psychological Bulletin, 72, 77–94.
239. Rescorla, R. A. (1970). Reduction in effectiveness of reinforcement after prior excitatory conditioning. Learning and Motivation, 1, 372–381.
240. Rescorla, R. A. (1972). “Configural” conditioning in discretetrial bar pressing. Journal of Comparative and Physiological Psychology, 79, 307–317.
241. Rescorla, R. A. (1973a). Evidence for a “unique stimulus” account of configural conditioning. Journal of Comparative and Physiological Psychology, 85, 331–338.
242. Rescorla, R. A. (1973b). Informational variables in Pavlovian conditioning. In G. Bower (Ed.), The psychology of learning and motivation (Vol. 6, pp. 1–46). New York: Academic Press.
243. Rescorla,R.A.(1980).Pavloviansecond-orderconditioning:Studies in associative learning. Hillsdale, NJ: Erlbaum.
244. Rescorla, R. A. (1985). Conditioned inhibition and facilitation. In R. R. Miller & N. E. Spear (Eds.) Information processing in animals: Conditioned inhibition (pp. 299–326). Hillsdale, NJ: Erlbaum.
245. Rescorla, R. A. (1988a). Behavioral studies of Pavlovian conditioning. Annual Review of Neuroscience, 11, 329–352.
246. Rescorla, R. A. (1988b). Pavlovian conditioning: It’s not what you think it is. American Psychologist, 43(3), 151–160.
247. Rescorla, R. A. (1991). Separate reinforcement can enhance the effectiveness of modulators. Journal of Experimental Psychology: Animal Behavior Processes, 17, 259–269.
248. Rescorla, R. A. (1993). Preservation of response-outcome associations through extinction. Animal Learning and Behavior, 21, 238–245.
249. Rescorla, R. A. (1997). Summation: Test of a configural theory. Animal Learning and Behavior, 25, 200–209.
250. Rescorla, R. A. (2000). Associative changes in excitors and inhibitors differ when they are conditioned in compound. Journal of Experimental Psychology: Animal Behavior Processes, 26, 428–438.
251. Rescorla, R. A., & Freberg, L. (1978). The extinction of withincompound flavor associations. Learning and Motivation, 9, 411– 427.
252. Rescorla, R. A., & Furrow, D. R. (1977). Stimulus similarity as a determinant of Pavlovian conditioning. Journal of Experimental Psychology: Animal Behavior Processes, 3, 203–215.
253. Rescorla, R. A., & Holland, P. C. (1976). Some behavioral approaches to the study of learning. In E. Bennett & M. R. Rozensweig (Eds.), Neural mechanisms of learning and memory (pp. 165–192). Cambridge, MA: MIT Press.
254. Rescorla, R.A., & Holland, P. C. (1982). Behavioral studies of associative learning in animals. Annual Review of Psychology, 33, 265–308.
255. Rescorla, R. A., & Solomon, R. L. (1967). Two process learning theory: Relationships between classical conditioning and instrumental learning. Psychological Review, 74, 151–182.
256. Rescorla, R. A., & Wagner, A. R. (1972). A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and nonreinforcement. In A. H. Black & W. F. Prokasy (Eds.), Classical conditioning (Vol. 2, pp. 64–99). New York: AppletonCentury-Crofts.
257. Rudy, J. W., & Sutherland, R. J. (1995). Configural association theory and the hippocampal formation: An appraisal and reconfiguration. Hippocampus, 5, 375–389.
258. Savastano,H.I.,&Miller,R.R.(1998).TimeascontentinPavlovian conditioning. Behavioural Processes, 44, 147–162.
259. Scharff, C., & Nottebohm, F. (1991). A comparative study of the behavioral deficits following lesions of various parts of the zebra finch song system: Implications for vocal learning. Journal of Neuroscience, 11, 2896–2913.
260. Schiller, C. H. (1957). Instinctive behavior. New York: International Universities Press.
261. Schmajuk, N. A., & DiCarlo, J. J. (1991). A neural network approach to hippocampal function in classical conditioning. Behavioral Neuroscience, 105, 82–110.
262. Schmajuk, N. A., & Holland, P. C. (1998). Occasion setting: Associative learning and cognition in animals. Washington, DC: American Psychological Association.
263. Schmajuk, N. A., Lamoureux, J. A., & Holland, P. C. (1998). Occasion setting: A neural network approach. Psychological Review, 105, 3–32.
264. Schmidt, M. F., & Konishi, M. (1998). Gating of auditory responses in the vocal control system of awake songbirds. Nature Neuroscience, 1, 513–518.
265. Schneirla, T. C. (1956). Interrelationships of the “innate” and the “acquired” in instinctive behavior. In P. Grasse (Ed.), L’Instinct dans le compartement des animaux et de l’homme (pp. 387– 452). Paris: Masson.
266. Schoenbaum, G., Chiba, A. A., & Gallagher, M. (1998). Orbitofrontral cortex and basolateral amygdala encode expected outcomes during learning. Nature Neuroscience, 1, 155–159.
267. Searcy, W. A., & Andersson, M. (1986). Sexual selection and the evolution of song. Annual Review of Ecology Systems, 17, 507– 533.
268. Sechenov, I. M. (1965). Reflexes of the brain (S. Belsky, Trans.). Cambridge, MA: MIT Press. (Original work published 1863).
269. Seligman, M. E. P. (1970). On the generality of the laws of learning. Psychological Review, 77, 406–418.
270. Seligman, M. E. P., Bravman, S., & Radford, R. (1970). Drinking: Discriminative conditioning in the rat. Psychonomic Science, 20, 63–64.
271. Setlow, B., Holland, P. C., & Gallagher, M. (2002). Disconnection of the basolateral amygdala complex and nucleus accumbens impairs appetitive second-order conditioned responses. Behavioral Neuroscience, 116, 267–275.
272. Sheffield, F. D. (1965). Relation between classical conditioning and instrumental learning. In W. F. Prokasy (Ed.), Classical conditioning (pp. 302–322). New York: Appleton-Century-Crofts.
273. Sheffield, F. D., & Campbell, B. A. (1954). The role of experience in the “spontaneous” activity of hungry rats. Journal of Comparative and Physiological Psychology, 47, 97–100.
274. Sheffield, F. D., & Roby, T. B. (1950). Reward value of a nonnutritive sweet taste. Journal of Comparative and Physiological Psychology, 43, 471–481.
275. Sherrington, C. S. (1906). The integrative action of the nervous system. London: A. Constable.
276. Shettleworth, S. J. (1998). Cognition, evolution, and behavior. New York: Oxford University Press.
277. Shettleworth, S. J., & Hampton, R. H. (1998). Adaptive specialization in food storing birds? Approaches to testing a comparative hypothesis.InI.Pepperberg,R.Balda,&A.Kamil(Eds.),Animal cognition in nature (pp. 65–98). San Diego, CA:Academic Press.
278. Silva, K. M., & Timberlake, W. (1998). The organization and temporal properties of appetitive behavior in rats. Animal Learning and Behavior, 26, 182–195.
279. Silva, K. M., & Timberlake, W. (2000). A clarification of the nature of backward excitatory conditioning. Learning and Motivation, 31, 67–80.
280. Skinner, B. F. (1957). Verbal behavior. New York: AppletonCentury-Crofts.
281. Skinner, D. M., Goddard, M. J., & Holland, P. C. (1998). What can nontraditional features tell us about conditioning and occasion setting? In N. A. Schmajuk & P. C. Holland (Eds.), Occasion setting: Associative learning and cognition in animals (pp. 113–144). Washington, DC: American Psychological Association.
282. Slater, P. J. B. (1989). Bird song learning: Causes and consequences. Ethology, Ecology, and Evolution, 1, 19–46.
283. Slater, P. J. B., Jones, A., & Ten Cate, C. (1993). Can lack of experience delay the end of the sensitive period for song learning? Netherlands Journal of Zoology, 43, 80–90.
284. Smith, M. C., Coleman, S. R., & Gormezano, I. (1969). Classical conditioning of the rabbit’s nictitating membrane response at backward, simultaneous, and forward CS-US intervals. Journal of Comparative and Physiological Psychology, 66, 679–687.
285. Soha, J. A., & Marler, P. (2001). Vocal syntax development in the white-crowned sparrow (Zonotrichia leucophrys). Journal of Comparative Psychology, 115, 172–180.
286. Solis, M. M., & Doupe, A. J. (1997). Anterior forebrain neurons develop selectivity by an intermediate stage of birdsong learning. Journal of Neuroscience, 17, 6447–6462.
287. Stripling, R., Volman, S. F., & Clayton, D. F. (1997). Response modulation in the zebra finch neostriatum: Relationship to nuclear gene regulation. Journal of Neuroscience, 17, 3883–3893.
288. Sutherland, N. S., & Mackintosh, N. J. (1971). Mechanisms of animal discrimination learning. New York: Academic Press.
289. Suthers, R. A. (1997). Peripheral control and lateralization of birdsong. Journal of Neurobiology, 33, 632–652.
290. Suzuki, W. A., & Clayton, N. S. (2000). The hippocampus and memory: A comparative and ethological perspective. Current Opinion in Neurobiology, 10, 768–773.
291. Swartzentruber, D. (1995). Modulatory mechanisms in Pavlovian conditioning. Animal Learning and Behavior, 23, 123–143.
292. Swithers, S. E., & Hall, W. G. (1994). Does oral experience terminate ingestion? Appetite, 23, 113–138.
293. Teitelbaum, P., & Pellis, P. M. (1992). Toward a synthetic physiological psychology. Psychological Science, 3, 4–20.
294. Testa, T. J. (1975). Effects of similarity of location and temporal intensity pattern of conditioned suppression in rats. Journal of Experimental Psychology: Animal Behavior Processes, 1, 114– 121.
295. Thorndike, E. L. (1898). Animal intelligence: An experimental study of the associative processes in animals. Psychological Monographs, 2(4, Whole No. 8).
296. Thorpe, W. H. (1958). The learning of song patterns by birds, with especial reference to the song of the chaffinch, Fingilla coelebs. Ibis, 100, 535–570.
297. Thorpe, W. H. (1961). Bird-song. Cambridge, UK: Cambridge University Press.
298. Thorpe, W. H. (1963). Learning and instinct in animals. London: Methuen.
299. Thorpe, W. H. (1979). The origins and rise of ethology. New York: Praeger.
300. Tierney, A. J. (1986). The evolution of learned and innate behavior: Contributions from genetics and neurobiology to a theory of behavioral evolution. Animal Learning and Behavior, 14(4), 339– 348.
301. Timberlake, W. (1983). The functional organization of appetitive behavior: Behavior systems and learning. In M. D. Zeiler & P. Harzem (Eds.), Advances in the analysis of behavior: Vol. 3. Biological factors in learning (pp. 177–221). Chichester, UK: Wiley.
302. Timberlake, W. (1994). Behavior systems, associationism, and Pavlovian conditioning. Psychonomic Bulletin and Review, 1, 405–420.
303. Timberlake, W., & Grant, D. L. (1975). Autoshaping in rats to the presentation of another rat predicting food. Science, 190, 690– 692.
304. Timberlake, W., & Silva, K. M. (1995). Appetitive behavior in ethology, psychology, and behavior systems. In N. S. Thompson (Ed.), Perspectives in ethology: Vol. 11. Behavioral design (pp. 211–253). New York: Plenum Press.
305. Timberlake,W.,Wahl, G., & King, D. (1982). Stimulus and response contingencies in the misbehavior of rats. Journal of Experimental Psychology: Animal Behavior Processes, 8, 62–85.
306. Tinbergen, N. (1951). The study of instinct. Oxford, UK: Clarendon Press.
307. Tinbergen, N. (1963). On aims and methods of ethology. Zeitschrift für Tierpsychologie, 20, 410–433.
308. Tolman, E. C., & Brunswik, E. (1935). The organism and the causal texture of the environment. Psychological Review, 42, 43–77.
309. Trapold, M. A. (1970). Are expectancies based upon different positive reinforcing events discriminably different? Learning and Motivation, 1, 129–140.
310. Vates, G. E., Broome, B. M., Mello, C. V., & Nottebohm, F. (1996). Auditory pathways of caudal telencephalon and their relation to the song system of adult male zebra finches (Taenopygia guttata). Journal of Comparative Neurology, 366, 613–642.
311. Vicario, D. S. (1991). Organization of the zebra finch song control system: Pt. 2. Functional organization of outputs from nucleus Robustus archistriatalis. Journal of Comparative Neurology, 309, 486–494.
312. Vicario, D. S., & Nottebohm, F. (1988). Organization of the zebra finch song control system: Representation of syringeal muscles in the hypoglossal nucleus. Journal of Comparative Neurology, 271, 346–354.
313. Vu, E. T., Mazurek, M. E., & Kuo, Y.-C. (1994). Identification of a forebrain motor programming network for the learned song of zebra finches. Journal of Neuroscience, 14, 6924–6934.
314. Wagner,A. R.(1978).Expectancies andthepriming of STM. In S. H. Hulse, H. Fowler, & W. K. Honig (Eds.), Cognitive processes in animal behavior (pp. 177–209). Hillsdale, NJ: Erlbaum.
315. Weingarten, H. P. (1983). Conditioned cues elicit feeding in sated rats: A role for learning in meal initiation. Science, 220, 431– 433.
316. Welty, J. C., & Baptista, L. (1988). The life of birds. New York: Saunders College.
317. West, M. J., & King, A. P. (1996). Eco-gen-actics: A systems approach to ontogeny of avian communication. In D. E. Kroodsma & E. H. Miller (Eds.), Ecology and evolution of acoustic communication in birds (pp. 20–38). Ithaca, NY: Cornell University Press.
318. West, M. J., King, A. P., & Duff, M. A. (1990). Communicating about communicating: When innate is not enough. Developmental Psychobiology, 23, 585–595.
319. Whaling, C. S., Nelson, D. A., & Marler, P. R. (1995). Testosteroneinduced shortening of the storage phase of song development in birds interferes with vocal learning. Developmental Psychobiology, 28, 367–376.
320. Wild, J. M. (1997). Neural pathways for the control of birdsong production. Journal of Neurobiology, 33, 653–670.
321. Yeo, A. G. (1974). The acquisition of conditioned suppression as a function of interstimulus interval duration. Quarterly Journal of Experimental Psychology, 26, 405–416.
322. Yu, A. C., & Margoliash, D. (1996). Temporal hierarchical control of singing in birds. Science, 273, 1871–1875.
323. Zamble, E. (1973). Augmentation of eating following a signal for feeding in rats. Learning and Motivation, 4, 138–147.
324. Zimmer-Hart, C. L., & Rescorla, R. A. (1974). Extinction of Pavlovian conditioned inhibition. Journal of Comparative and Physiological Psychology, 86, 837–845.