Learning theory focuses on the thought processes that underlie learning

l e a r n i n g

5 ch

ap te

r

160

fel77023_ch05_160-197.indd 160 11/13/08 12:06:29 PM

161

Declan lies on his back wanting his belly scratched.

The eight-year-old black Labrador cross swings his legs

in the air for a few minutes before resigning himself to

chewing on someone’s shoe.

In the office he behaves like any pet dog, but in the

field he is like a tornado—focused on finding illegal

drugs being smuggled. Declan is a drug-detector dog

for the Customs Service and has been busting drug

smugglers with his handler, Kevin Hattrill, for eight

years.

Airport passengers look on with curiosity as Declan

darts around people and their luggage. Within minutes

he sniffs out a person of interest, who is taken away

and questioned by airport authorities.

Dogs like Declan are trained to detect illegal drugs,

such as cannabis, methamphetamine, and cocaine, or

explosives. Hattrill said the dogs were dual response-

trained when they detected something. “If the odor

is around a passenger, they are trained to sit beside

them. If it’s around cargo, they are trained to scratch.

When they detect something, their whole tempera-

ment will change.

“The dogs can screen up to 300 people within 10 to

15 minutes at the airport. Nothing else can do that.”

(McKenzie-McLean, 2006, p. 7)

A Four-Legged Co-Worker

Declan’s expertise did not just happen, of course. It is the result of painstaking training procedures—the same ones that are at work in each of our lives, illustrated by our ability to read a book, drive a car, play poker, study for a test, or perform any of the numerous activities that make up our daily routine. Like Declan, each of us must acquire and then refine our skills and abilities through learning.

Learning is a fundamental topic for psychologists and plays a central role in almost every specialty area of psychology. For example, a developmental psychologist might inquire, “How do babies learn to distinguish their mothers from other people?” whereas a clinical psychologist might wonder, “Why do some people learn to be afraid when they see a spider?”

Psychologists have approached the study of learning from several angles. Among the most fundamen- tal are studies of the type of learning that is illustrated in responses ranging from a dog salivating when it hears its owner opening a can of dog food to the emotions we feel when our national anthem is played. Other theories consider how learning is a consequence of rewarding circumstances. Finally, several other approaches focus on the cognitive aspects of learning, or the thought processes that underlie learning.

ah ea

d

lo ok

in g

chapter outline

module 15 Classical Conditioning The Basics of Classical Conditioning

Applying Conditioning Principles to Human Behavior

Extinction

Generalization and Discrimination

module 16 Operant Conditioning The Basics of Operant Conditioning

Positive Reinforcers, Negative Reinforcers, and Punishment

The Pros and Cons of Punishment: Why Reinforcement Beats Punishment

Schedules of Reinforcement: Timing Life’s Rewards

Shaping: Reinforcing What Doesn’t Come Naturally

Becoming an Informed Consumer of Psychology: Using Behavior Analysis and Behavior Modification

module 17 Cognitive Approaches to Learning Latent Learning

Observational Learning: Learning Through Imitation

Violence in Television and Video Games: Does the Media’s Message Matter?

Exploring Diversity: Does Culture Influence How We Learn?

Psychology on the Web The Case of . . . The Manager Who Doubled Productivity Full Circle: Learning

fel77023_ch05_160-197.indd 161 11/13/08 12:01:39 PM

162 Chapter 5 learning

Classical Conditioning

module 15

learning outcomes 15.1 Describe the basics of classical conditioning and how they relate to learning.

15.2 Give examples of applying conditioning principles to human behavior.

15.3 Explain extinction.

15.4 Discuss stimulus generalization and discrimination.

Does the mere sight of the golden arches in front of McDonald’s make you feel pangs of hunger and think about hamburgers? If it does, you are displaying an elementary form of learning called clas- sical conditioning. Classical conditioning helps explain such diverse phenomena as crying at the sight of a bride walking down the aisle, fearing the dark, and falling in love.

Classical conditioning is one of a number of different types of learning that psychologists have identified, but a general definition encompasses them all: learning is a relatively permanent change in behavior that is brought about by experience.

We are primed for learning from the beginning of life. Infants exhibit a primitive type of learning called habituation. Habituation is the decrease in response to a stimulus that occurs after repeated presentations of the same stimulus. For example, young infants may initially show interest in a novel stimulus, such as a brightly colored

toy, but they will soon lose interest if they see the same toy over and over. (Adults exhibit habituation, too: newlyweds soon stop noticing that they are wearing a wedding ring.) Habituation permits us to ignore things that have stopped providing new information. Most learning is considerably more complex than habituation, and the study

of learning has been at the core of the field of psychology. Although philoso- phers since the time of Aristotle have speculated on the foundations of learn- ing, the first systematic research on learning was done at the beginning of the twentieth century, when Ivan Pavlov (does the name ring a bell?) developed the framework for learning called classical conditioning.

The Basics of Classical Conditioning In the early twentieth century, Ivan Pavlov, a famous Russian physiologist, had been studying the secretion of stomach acids and salivation in dogs in response to the ingestion of varying amounts and kinds of food. While doing that he observed a curious phenomenon: sometimes stomach secretions and salivation would begin in the dogs when they had not yet eaten any food. The mere sight of the experimenter who normally brought the food, or even the sound of the experimenter’s footsteps, was enough to produce salivation in the dogs.

Learning A relatively permanent change in behavior brought about by experience.

Learning A relatively permanent change in behavior brought about by experience.

LO 1LO 1

fel77023_ch05_160-197.indd 162 11/13/08 12:01:48 PM

Module 15 classical conditioning 163

Pavlov’s genius lay in his ability to recognize the implications of this discov- ery. He saw that the dogs were responding not only on the basis of a biological need (hunger), but also as a result of learning—or, as it came to be called, classi- cal conditioning. Classical conditioning is a type of learning in which a neu- tral stimulus (such as the experimenter’s footsteps) comes to elicit a response after being paired with a stimulus (such as food) that naturally brings about that response.

To demonstrate classical conditioning, Pavlov (1927) attached a tube to the salivary gland of a dog, allowing allow him to measure precisely the dog’s salivation. He then rang a bell and, just a few seconds later, pre- sented the dog with meat. This pairing occurred repeatedly and was care- fully planned so that, each time, exactly the same amount of time elapsed between the presentation of the bell and the meat. At first the dog would salivate only when the meat was presented, but soon it began to salivate at the sound of the bell. In fact, even when Pavlov stopped presenting the meat, the dog still salivated after hearing the sound. The dog had been classically conditioned to salivate to the bell.

As you can see in Figure 1 , the basic processes of classical conditioning that underlie Pavlov’s discovery are straightforward, although the termi- nology he chose is not simple. Consider first the diagram in Figure 1A . Before conditioning, there are two unrelated stimuli: the ringing of a bell and meat. We know that normally the ringing of a bell does not lead to salivation but to some irrelevant response, such as pricking up the ears or perhaps a startle reaction. The bell is therefore called the neutral stimulus because it is a stimulus that, before conditioning, does not naturally bring about the response in which we are interested. We also have meat, which naturally causes a dog to salivate—the response we are interested in condi- tioning. The meat is considered an unconditioned stimulus, or UCS, because food placed in a dog’s mouth automatically causes salivation to occur. The response that the meat elicits (salivation) is called an unconditioned response, or UCR —a natural, innate, reflexive response that is not associated with previous learning. Unconditioned responses are always brought about by the presence of unconditioned stimuli.

Figure 1B illustrates what happens during conditioning. The bell is rung just before each presentation of the meat. The goal of conditioning is for the dog to associate the bell with the unconditioned stimulus (meat) and therefore to bring about the same sort of response as the unconditioned stimulus. After a number of pairings of the bell and meat, the bell alone causes the dog to salivate.

Classical conditioning A type of learning in which a neutral stimulus comes to bring about a response after it is paired with a stimulus that naturally brings about that response.

Neutral stimulus A stimulus that, before conditioning, does not naturally bring about the response of interest.

Unconditioned stimulus (UCS) A stimulus that naturally brings about a particular response without having been learned.

Unconditioned response (UCR) A response that is natural and needs no training (e.g., salivation at the smell of food).

Classical conditioning A type of learning in which a neutral stimulus comes to bring about a response after it is paired with a stimulus that naturally brings about that response.

Neutral stimulus A stimulus that, before conditioning, does not naturally bring about the response of interest.

Unconditioned stimulus (UCS) A stimulus that naturally brings about a particular response without having been learned.

Unconditioned response (UCR) A response that is natural and needs no training (e.g., salivation at the smell of food).

Ivan Pavlov (center) developed the principles of classical conditioning.

study alert Figure 1 (on the next page)

can help you learn and understand the process (and

terminology) of classical conditioning, which can be

confusing.

fel77023_ch05_160-197.indd 163 11/13/08 12:01:49 PM

164 Chapter 5 learning

When conditioning is complete, the bell has evolved from a neutral stimulus to what is now called a conditioned stimulus, or CS. At this time, salivation that occurs as a response to the conditioned stimulus (bell) is considered a conditioned response, or CR. This situation is depicted in Figure 1C . After conditioning, then, the conditioned stimulus evokes the conditioned response.

The sequence and timing of the presenta- tion of the unconditioned stimulus and the conditioned stimulus are particularly impor- tant. Like a malfunctioning warning light at a railroad crossing that goes on after the train has passed by, a neutral stimulus that follows an unconditioned stimulus has little chance of becoming a conditioned stimu- lus. However, just as a warning light works best if it goes on right before a train passes, a neutral stimulus that is presented just before the unconditioned stimulus is most apt to result in successful conditioning (Bitterman, 2006).

Although the terminology Pavlov used to describe classical conditioning may seem confusing, the following summary can help make the relationships between stimuli and responses easier to understand and remember:

Before Conditioning

During Conditioning

Neutral stimulus

After Conditioning

Sound of bell Pricking of

ears

Unconditioned stimulus (UCS)

Conditioned stimulus (CS)

Meat

Meat

Salivation

Salivation

Salivation

Sound of bell

Sound of bell

Neutral stimulus

Unconditioned stimulus (UCS)

Conditioned response (CR)

Unconditioned response (UCR)

Unconditioned response (UCR)

Response unrelated to meat

A

B

C

Figure 1 The basic process of classical conditioning. (A) Before conditioning, the ringing of a bell does not bring about salivation—making the bell a neutral stimulus. In contrast, meat naturally brings about salivation, making the meat an unconditioned stimulus and salivation an unconditioned response. (B) During conditioning, the bell is rung just before the presentation of the meat. (C) Eventually, the ringing of the bell alone brings about salivation. We now can say that conditioning has been accomplished: the previously neutral stimulus of the bell now is a conditioned stimulus that brings about the conditioned response of salivation.

■ Conditioned � learned. ■ Unconditioned � not learned. ■ An un conditioned stimulus leads to an

un conditioned response. ■ Un conditioned stimulus– un conditioned

response pairings are un learned and un trained.

■ During conditioning, a previously neu- tral stimulus is transformed into the conditioned stimulus.

■ A conditioned stimulus leads to a con- ditioned response, and a conditioned stimulus–conditioned response pair- ing is a consequence of learning and training.

■ An unconditioned response and a con- ditioned response are similar (such as salivation in Pavlov’s experiment), but the unconditioned response occurs naturally, whereas the conditioned response is learned.

fel77023_ch05_160-197.indd 164 11/13/08 12:01:56 PM

Module 15 classical conditioning 165

Applying Conditioning Principles to Human Behavior Although the initial conditioning experiments were carried out with ani- mals, classical conditioning principles were soon found to explain many aspects of everyday human behavior. Recall, for instance, the earlier illus- tration of how people may experience hunger pangs at the sight of McDon- ald’s golden arches. The cause of this reaction is classical conditioning: the previously neutral arches have become associated with the food inside the restaurant (the unconditioned stimulus), causing the arches to become a conditioned stimulus that brings about the conditioned response of hunger.

Emotional responses are especially likely to be learned through classi- cal conditioning processes. For instance, how do some of us develop fears of mice, spiders, and other creatures that are typically harmless? In a now infamous case study, psychologist John B. Watson and colleague Rosalie Rayner (1920) showed that classical conditioning was at the root of such fears by condi- tioning an 11-month-old infant named Albert to be afraid of rats. “Little Albert,” like most infants, initially was frightened by loud noises but had no fear of rats.

In the study, the experimenters sounded a loud noise just as they showed Little Albert a rat. The noise (the unconditioned stimulus) evoked fear (the unconditioned response). However, after just a few pairings of noise and rat, Albert began to show fear of the rat by itself, bursting into tears when he saw it. The rat, then, had become a CS that brought about the CR, fear. Furthermore, the effects of the conditioning lingered: five days later, Albert reacted with fear not only when shown a rat, but when shown objects that looked similar to the white, furry rat, including a white rabbit, a white sealskin coat, and even a white Santa Claus mask. (By the way, we don’t know what happened to the unfortunate Little Albert. Watson, the experimenter, has been condemned for using ethically questionable procedures that could never be conducted today.)

Learning by means of classical conditioning also occurs during adulthood. For example, you may not go to a dentist as often as you should because of prior associations of dentists with pain. On the other hand, classical conditioning also accounts for pleasant experiences. For instance, you may have a particular fond- ness for the smell of a certain perfume or aftershave lotion because the feelings and thoughts of an early love come rushing back whenever you encounter it. Classical conditioning, then, explains many of the reactions we have to stimuli in the world around us.

Extinction What do you think would happen if a dog that had become classically condi- tioned to salivate at the ringing of a bell never again received food when the bell was rung? The answer lies in one of the basic phenomena of learning: extinc- tion. Extinction occurs when a previously conditioned response decreases in frequency and eventually disappears.

To produce extinction, one needs to end the association between condi- tioned stimuli and unconditioned stimuli. For instance, if we had trained a dog to salivate (the conditioned response) at the ringing of a bell (the conditioned

LO 2LO 2 Conditioned stimulus (CS) A once- neutral stimulus that has been paired with an unconditioned stimulus to bring about a response formerly caused only by the unconditioned stimulus.

Conditioned response (CR) A response that, after conditioning, follows a previously neutral stimulus (e.g., salivation at the ringing of a bell).

Extinction A basic phenomenon of learning that occurs when a previously conditioned response decreases in frequency and eventually disappears.

Conditioned stimulus (CS) A once- neutral stimulus that has been paired with an unconditioned stimulus to bring about a response formerly caused only by the unconditioned stimulus.

Conditioned response (CR) A response that, after conditioning, follows a previously neutral stimulus (e.g., salivation at the ringing of a bell).

Extinction A basic phenomenon of learning that occurs when a previously conditioned response decreases in frequency and eventually disappears.

LO 3LO 3

Emotional responses are especially likely to be learned through classical conditioning processes.

fel77023_ch05_160-197.indd 165 11/13/08 12:02:00 PM

166 Chapter 5 learning

stimulus), we could produce extinction by repeatedly ringing the bell but not providing meat. At first the dog would continue to salivate when it heard the bell, but after a few such instances, the amount of salivation would probably decline, and the dog would eventually stop responding to the bell altogether. At that point, we could say that the response had been extinguished. In sum,

extinction occurs when the conditioned stimulus is presented repeatedly with- out the unconditioned stimulus (see Figure 2 ).

Once a conditioned response has been extin- guished, has it vanished forever? Not necessar- ily. Pavlov discovered this phenomenon when he returned to his dog a few days after the conditioned behavior had seemingly been extinguished. If he rang a bell, the dog once again salivated—an effect known as spontaneous recovery, or the reemer-

gence of an extinguished conditioned response after a period of rest and with no further conditioning.

Spontaneous recovery helps explain why it is so hard to overcome drug addictions. For example, cocaine addicts who are thought to be “cured” can experience an irresistible impulse to use the drug again if they are subsequently confronted by a stimulus with strong connections to the drug, such as a white powder (DiCano & Everitt, 2002; Rodd et al., 2004; Plowright, Simonds, & Butler, 2006).

Strong

Acquisition (conditioned response and unconditioned response presented together)

Extinction (conditioned stimulus by itself ) Spontaneous recovery of

conditioned response

Extinction follows (conditioned stimulus alone)

Training CS alone Pause Spontaneous recovery

St re

ng th

o f

co nd

iti on

ed r

es po

ns e

(C R

)

Weak

Time A B C D

Figure 2 Acquisition, extinction, and spontaneous recovery of a classically conditioned response. A conditioned response (CR) gradually increases in strength during training (A). However, if the conditioned stimulus is presented by itself enough times, the conditioned response gradually fades, and extinction occurs (B). After a pause (C) in which the conditioned stimulus is not presented, spontaneous recovery can occur (D). However, extinction typically reoccurs soon after.

Once a conditioned response has been extinguished, has it vanished forever?

Not necessarily.

Spontaneous recovery The reemergence of an extinguished conditioned response after a period of rest and with no further conditioning.

A Veterinary Assistant How might knowledge of classical conditioning

be useful in your career?

From the perspective of . . .

fel77023_ch05_160-197.indd 166 11/13/08 12:02:01 PM

Module 15 classical conditioning 167

Generalization and Discrimination Despite differences in color and shape, to most of us a rose is a rose is a rose. The pleasure we experience at the beauty, smell, and grace of the flower is similar for different types of roses. Pavlov noticed a similar phenomenon. His dogs often salivated not only at the ringing of the bell that was used during their original conditioning but at the sound of a buzzer as well.

Such behavior is the result of stimulus gener- alization. Stimulus generalization occurs when a conditioned response follows a stimulus that is similar to the original conditioned stimulus. The greater the similarity between two stimuli, the greater the likelihood of stimulus generalization. Little Albert, who, as we mentioned earlier, was conditioned to be fearful of white rats, grew afraid of other furry white things as well. However, according to the principle of stimulus generalization, it is unlikely that he would have been afraid of a black dog, because its color would have differentiated it sufficiently from the original fear-evoking stimulus.

On the other hand, stimulus discrimination occurs if two stimuli are sufficiently distinct from each other that one evokes a conditioned response but the other does not. Stimulus discrimination provides the ability to dif- ferentiate between stimuli. For example, my dog, Cleo, comes running into the kitchen when she hears the sound of the electric can opener, which she has learned is used to open her dog food when her dinner is about to be served. She does not bound into the kitchen at the sound of the food proces- sor, although it sounds similar. In other words, she discriminates between the stimuli of can opener and food pro- cessor. Similarly, our ability to discrimi- nate between the behavior of a growling dog and that of one whose tail is wagging can lead to adaptive behavior—avoiding the growling dog and petting the friendly one.

LO 4LO 4

Stimulus generalization Occurs when a conditioned response follows a stimulus that is similar to the original conditioned stimulus; the more similar the two stimuli are, the more likely generalization is to occur.

Stimulus discrimination The process that occurs if two stimuli are sufficiently distinct from each other that one evokes a conditioned response but the other does not; the ability to differentiate between stimuli.

Stimulus generalization Occurs when a conditioned response follows a stimulus that is similar to the original conditioned stimulus; the more similar the two stimuli are, the more likely generalization is to occur.

Stimulus discrimination The process that occurs if two stimuli are sufficiently distinct from each other that one evokes a conditioned response but the other does not; the ability to differentiate between stimuli.

The greater the similarity between two stimuli, the greater the likelihood of stimulus generalization.

Because of a previous unpleasant experience, a person may expect a similar occurrence when faced with a comparable situation in the future, a process known as stimulus generalization. Can you think of ways this process is used in everyday life?

study alert Remember that stimulus generalization relates to

stimuli that are similar to one another, while stimulus

discrimination relates to stimuli that are different

from one another.

fel77023_ch05_160-197.indd 167 11/26/08 10:49:29 PM

168 Chapter 5 learning

r e c a p Describe the basics of classical conditioning and how they relate to learning.

■ One major form of learning is classical condi- tioning, which occurs when a neutral stimu- lus—one that normally brings about no relevant response—is repeatedly paired with a stimulus (called an unconditioned stimulus) that brings about a natural, untrained response. (p. 163)

■ After repeated pairings, the neutral stimulus elicits the same response that the uncon- ditioned stimulus brings about. When this occurs, the neutral stimulus has become a conditioned stimulus, and the response a conditioned response. (p. 164)

Give examples of applying conditioning principles to human behavior.

■ Examples of classical conditioning include the development of emotions and fears. (p. 165)

Explain extinction.

■ Learning is not always permanent. Extinction occurs when a previously learned response decreases in frequency and eventually disappears. (p. 166)

Discuss stimulus generalization and discrimination.

■ Stimulus generalization is the tendency for a conditioned response to follow a stimulus that is similar to, but not the same as, the original conditioned stimulus. The converse phenom- enon, stimulus discrimination, occurs when an organism learns to distinguish between stimuli. (p. 167)

e v a l u a t e 1. involves changes brought about by experience. 2. is the name of the scientist responsible for discovering the learning phenomenon

known as conditioning, in which an organism learns a response to a stimulus to which it normally would not respond.

Refer to the passage below to answer questions 3 through 5: The last three times little Theresa visited Dr. Lopez for checkups, he administered a painful preventive immunization shot that left her in tears. Today, when her mother takes her for another checkup, Theresa begins to sob as soon as she comes face-to-face with Dr. Lopez, even before he has a chance to say hello.

3. The painful shot that Theresa received during each visit was a(n) that elicited the , her tears.

4. Dr. Lopez is upset because his presence has become a for Theresa’s crying. 5. Fortunately, Dr. Lopez gave Theresa no more shots for quite some time. Over that period she gradually

stopped crying and even came to like him. had occurred.

fel77023_ch05_160-197.indd 168 11/13/08 12:02:31 PM

Module 15 classical conditioning 169

Answers to Evaluate Questions 1. learning; 2. Pavlov, classical; 3. unconditioned stimulus, unconditioned response; 4. conditioned stimulus; 5. extinction

k e y t e r m s Learning p. 162

Classical conditioning p. 163

Neutral stimulus p. 163

Unconditioned stimulus (UCS) p. 163

Unconditioned response (UCR) p. 163

Conditioned stimulus (CS) p. 165

Conditioned response (CR) p. 165

Extinction p. 165

Spontaneous recovery p. 166

Stimulus generalization p. 167

Stimulus discrimination p. 167

r e t h i n k How likely is it that Little Albert, Watson’s experimental subject, went through life afraid of Santa Claus? Describe what could have happened to prevent his continual dread of Santa.

fel77023_ch05_160-197.indd 169 11/13/08 12:02:32 PM

170 Chapter 5 learning

module 16

Operant Conditioning

learning outcomes 16.1 Define the basics of operant conditioning.

16.2 Explain reinforcers and punishment.

16.3 Present the pros and cons of punishment.

16.4 Discuss schedules of reinforcement.

16.5 Explain the concept of shaping.

Very good . . . What a clever idea . . . Fantastic . . . I agree . . . Thank you . . . Excellent . . . Super . . . Right on . . . This is the best paper you’ve ever written; you get an A . . . You are really getting the hang of it . . . I’m impressed . . . You’re getting a raise . . . Have a cookie . . . You look great . . . I love you . . .

Few of us mind being the recipient of any of the preceding comments. But what is especially noteworthy about them is that each of these simple statements can be used, through a process known as operant conditioning, to bring about powerful changes in behavior and to teach the most complex tasks. Operant conditioning is the basis for many of the most important kinds of human, and animal, learning.

Operant conditioning is learning in which a voluntary response is strengthened or weakened, depending on its favorable or unfavor- able consequences. When we say that a response has been strength- ened or weakened, we mean that it has been made more or less likely

to recur regularly. Unlike classical conditioning, in which the original behaviors are the nat-

ural, biological responses to the presence of a stimulus such as food, water, or pain, operant conditioning applies to voluntary responses, which an organism performs deliberately to produce a desirable outcome. The term operant emphasizes this point: the organism operates on its environment to produce a desirable result. Operant conditioning is at work when we learn

that toiling industriously can bring about a raise or that exercising hard results in a good physique.

The Basics of Operant Conditioning The inspiration for a whole generation of psychologists studying operant con- ditioning was one of the twentieth century’s most influential psychologists, B. F. Skinner (1904–1990). Skinner was interested in specifying how behavior var- ies as a result of alterations in the environment.

Skinner conducted his research using an apparatus called the Skinner box (shown in Figure 1 ), a chamber with a highly controlled environment that was used to study operant conditioning processes with laboratory animals. Let’s consider what happens to a rat in the typical Skinner box (Pascual & Rodríguez, 2006).

Operant conditioning Learning in which a voluntary response is strengthened or weakened, depending on its favorable or unfavorable consequences.

Operant conditioning Learning in which a voluntary response is strengthened or weakened, depending on its favorable or unfavorable consequences.

LO 1LO 1

fel77023_ch05_160-197.indd 170 11/13/08 12:02:33 PM

Suppose you want to teach a hungry rat to press a lever that is in its box. At first the rat will wan- der around the box, exploring the environment in a relatively random fashion. At some point, however, it will probably press the lever by chance, and when it does, it will receive a food pellet. The first time this happens, the rat will not learn the connection between pressing a lever and receiving food and will continue to explore the box. Sooner or later the rat will press the lever again and receive a pellet, and in time the frequency of the pressing response will increase. Eventually, the rat will press the lever con- tinually until it satisfies its hunger, thereby demon- strating that it has learned that the receipt of food is contingent on pressing the lever.

Reinforcement: The Central Concept of Operant Conditioning Skinner called the process that leads the rat to con- tinue pressing the key “reinforcement.” Reinforcement is the process by which a stimulus increases the probability that a preceding behavior will be repeated. In other words, pressing the lever is more likely to occur again because of the stimulus of food.

In a situation such as this one, the food is called a reinforcer. A reinforcer is any stimulus that increases the probability that a preceding behavior will occur again. Hence, food is a reinforcer because it increases the probability that the behavior of pressing (formally referred to as the response of press- ing) will take place.

What kind of stimuli can act as reinforcers? Bonuses, toys, and good grades can serve as reinforcers—if they strengthen the probability of the response that occurred before their introduction.

There are two major types of reinforcers. A primary reinforcer satisfies some biological need and works naturally, regardless of a person’s prior experience. Food for a hungry person, warmth for a cold person, and relief for a person in pain all would be classified as primary reinforcers. A secondary reinforcer, in con- trast, is a stimulus that becomes reinforcing because of its association with a primary reinforcer. For instance, we know that money is valuable because we have learned that it allows us to obtain other desirable objects, including primary reinforcers such as food and shelter. Money thus becomes a secondary reinforcer.

Positive Reinforcers, Negative Reinforcers, and Punishment In many respects, reinforcers can be thought of in terms of rewards; both a reinforcer and a reward increase the probability that a preceding response will occur again. But the term reward is limited to positive occurrences, and this is where it differs from a reinforcer—for it turns out that reinforcers can be positive or negative.

Reinforcement The process by which a stimulus increases the probability that a preceding behavior will be repeated.

Reinforcer Any stimulus that increases the probability that a preceding behavior will occur again.

Reinforcement The process by which a stimulus increases the probability that a preceding behavior will be repeated.

Reinforcer Any stimulus that increases the probability that a preceding behavior will occur again.

LO 2LO 2

Module 16 operant conditioning 171

Food dispenser

Response lever

Figure 1 B. F. Skinner with a Skinner box used to study operant conditioning. Laboratory rats learn to press the lever in order to obtain food, which is delivered in the tray.

Bonuses, toys, and good grades can serve as reinforcers—if they strengthen the probability of the response that occurred before their introduction.

study alert Remember that primary

reinforcers satisfy a biological need; secondary

reinforcers are effective due to previous association with

a primary reinforcer.

fel77023_ch05_160-197.indd 171 11/13/08 12:02:34 PM

172 Chapter 5 learning

A positive reinforcer is a stimulus added to the environment that brings about an increase in a preceding response. If food, water, money, or praise is provided after a response, it is more likely that that response will occur again in the future. The paychecks that workers get at the end of the week, for example, increase the likelihood that they will return to their jobs the following week.

In contrast, a negative reinforcer refers to an unpleasant stimulus whose removal leads to an increase in the probability that a preceding response will be repeated in the future. For example, if you have an itchy rash (an unpleas- ant stimulus) that is relieved when you apply a certain brand of ointment, you are more likely to use that ointment the next time you have an itchy rash. Using the ointment, then, is negatively reinforcing, because it removes the unpleasant itch. Negative reinforcement, then, teaches the individual that taking an action removes a negative condition that exists in the envi- ronment. Like positive reinforcers, negative reinforcers increase the likeli- hood that preceding behaviors will be repeated.

Note that negative reinforcement is not the same as punishment. Punishment refers to a stimulus that decreases the probability that a prior behavior will occur again. Unlike negative reinforcement, which produces

an increase in behavior, punishment reduces the likelihood of a prior response. If we receive a shock that is meant to decrease a certain behavior, then, we are receiving punishment, but if we are already receiving a shock and do some- thing to stop that shock, the behavior that stops the shock is considered to be negatively reinforced. In the first case, the specific behavior is apt to decrease because of the punishment; in the second, it is likely to increase because of the negative reinforcement.

There are two types of punishment: positive punishment and negative pun- ishment, just as there are positive reinforcement and negative reinforcement. (In both cases, “positive” means adding something, and “negative” means removing something.) Positive punishment weakens a response through the application of an unpleasant stimulus. For instance, spanking a child for misbehaving, or spend- ing 10 years in jail for committing a crime, is positive punishment. In con- trast, negative punishment consists of the removal of something pleasant. For instance, when a teenager is told she is “grounded” and will no longer be able to use the family car because of her poor grades, or when an employee is informed that he has been demoted with a cut in pay because of a poor job evaluation, negative punishment is being administered. Both positive and negative punishment result in a decrease in the likelihood that a prior behavior will be repeated.

The following rules (and the summary in Figure 2 ) can help you distinguish these concepts from one another:

■ Reinforcement increases the frequency of the behavior pre- ceding it; punishment decreases the frequency of the behavior preceding it.

■ The application of a positive stimulus brings about an increase in the frequency of behavior and is referred to as positive

Positive reinforcer A stimulus added to the environment that brings about an increase in a preceding response.

Negative reinforcer An unpleasant stimulus whose removal leads to an increase in the probability that a preceding response will be repeated in the future.

Punishment A stimulus that decreases the probability that a previous behavior will occur again.

Positive reinforcer A stimulus added to the environment that brings about an increase in a preceding response.

Negative reinforcer An unpleasant stimulus whose removal leads to an increase in the probability that a preceding response will be repeated in the future.

Punishment A stimulus that decreases the probability that a previous behavior will occur again.

From the perspective of . . . A Retail Supervisor How might you use the principles of operant conditioning to

change employee behavior involving tardiness, customer service, or store cleanliness?

fel77023_ch05_160-197.indd 172 11/13/08 12:02:39 PM

reinforcement; the application of a negative stimulus decreases or reduces the frequency of behavior and is called punishment.

■ The removal of a negative stimulus that results in an increase in the frequency of behavior is negative reinforcement; the removal of a positive stimulus that decreases the frequency of behavior is negative punishment.

The Pros and Cons of Punishment: Why Reinforcement Beats Punishment Is punishment an effective way to modify behavior? Punishment often pres- ents the quickest route to changing behavior that, if allowed to continue, might be dangerous to an individual. For instance, a parent may not have a second chance to warn a child not to run into a busy street, and so punishing the first incidence of this behavior may prove to be wise. Moreover, the use of punishment to suppress behavior, even temporarily, provides an opportu- nity to reinforce a person for subsequently behav- ing in a more desirable way.

Punishment has several disadvantages that make its routine use questionable. For one thing, punishment is frequently ineffective, particu- larly if it is not delivered shortly after the undesired behavior or if the indi- vidual is able to leave the setting in which the punishment is being given. An

LO 3LO 3

Increase in behavior (reinforcement)

Decrease in behavior (punishment)

Intended Result

Example: Giving a raise for good performance

Example: Applying ointment to relieve an itchy rash leads to a higher future likelihood of applying the ointment

When Stimulus Is Added, the Result Is . . .

When Stimulus Is Removed or Terminated, the Result Is . . .

Positive Reinforcement Negative Reinforcement

Negative PunishmentPositive Punishment

Result: Increase in response of good performance

Example: Yelling at a teenager when she steals a bracelet

Result: Decrease in frequency of response of stealing

Example: Teenager’s access to car restricted by parents due to teenager’s breaking curfew

Result: Decrease in response of breaking curfew

Result: Increase in response of using ointment

Figure 2 Types of reinforcement and punishment.

Punishment has several disadvantages that make its routine use questionable.

study alert The differences between

positive reinforcement, negative reinforcement,

positive punishment, and negative punishment

are tricky, so pay special attention to Figure 2 and

the rules in the text.

Module 16 operant conditioning 173

fel77023_ch05_160-197.indd 173 11/26/08 10:49:36 PM

174 Chapter 5 learning

employee who is reprimanded by the boss may quit; a teenager who loses the use of the family car may borrow a friend’s car instead. In such instances, the initial behavior that is being punished may be replaced by one that is even less desirable.

Even worse, physical punishment can convey to the recipient the idea that physical aggression is permissible and perhaps even desirable. A father who yells at and hits his son for misbehaving teaches the son that aggression is an appropriate, adult response. The son soon may copy his father’s behavior by acting aggressively toward others. In addition, physical punishment is often administered by people who are themselves angry or enraged. It is unlikely that individuals in such an emotional state will be able to think through what they are doing or control carefully the degree of punishment they are inflicting (Baumrind, Larzelere, & Cowan, 2002; Sorbring, Deater-Deckard, & Palmerus, 2006).

In short, the research findings are clear: reinforcing desired behavior is a more appropriate technique for modifying behavior than using punishment (Hiby, Rooney, & Bradshaw, 2004; Sidman, 2006).

Schedules of Reinforcement: Timing Life’s Rewards The world would be a different place if poker players never played cards again after the first losing hand, fishermen returned to shore as soon as they missed a

catch, or telemarketers never made another phone call after their first hang- up. The fact that such unreinforced behaviors continue, often with great fre- quency and persistence, illustrates that reinforcement need not be received continually for behavior to be learned and maintained. In fact, behavior that is reinforced only occasionally can ultimately be learned better than can behavior that is always reinforced.

When we refer to the frequency and timing of reinforcement that fol- lows desired behavior, we are talking about schedules of reinforcement. Behavior that is reinforced every time it occurs is said to be on a continuous reinforcement schedule; if it is reinforced some but not all of the time, it is on a partial (or intermittent ) reinforcement schedule. Although learn- ing occurs more rapidly under a continuous reinforcement schedule, behav- ior lasts longer after reinforcement stops when it is learned under a partial

reinforcement schedule (Staddon & Cerutti, 2003; Gottlieb, 2004; Casey, Cooper-Brown, & Wacher, 2006).

Why should intermittent reinforcement result in stronger, longer-lasting learning than con- tinuous reinforcement? We can answer the ques- tion by examining how we might behave when using a candy vending machine compared with a Las Vegas slot machine. When we use a vend- ing machine, prior experience has taught us that every time we put in the appropriate amount of money, the reinforcement, a candy bar, ought to be delivered. In other words, the schedule of

LO 4LO 4

Schedules of reinforcement Different patterns of frequency and timing of reinforcement following desired behavior.

Continuous reinforcement schedule Reinforcing of a behavior every time it occurs.

Partial (or intermittent) reinforcement schedule Reinforcing of a behavior some but not all of the time.

Schedules of reinforcement Different patterns of frequency and timing of reinforcement following desired behavior.

Continuous reinforcement schedule Reinforcing of a behavior every time it occurs.

Partial (or intermittent) reinforcement schedule Reinforcing of a behavior some but not all of the time.

© T

he N

ew Y

or ke

r C

ol le

ct io

n, 2

0 0

1 C

. W ey

an t f

ro m

ca

rt oo

nb an

k. co

m . A

ll ri

gh ts

r es

er ve

d .

fel77023_ch05_160-197.indd 174 11/13/08 12:02:47 PM

reinforcement is continuous. In comparison, a slot machine offers intermittent reinforcement. We have learned that after putting in our cash, most of the time we will not receive anything in return. At the same time, though, we know that we will occasionally win something.

Now suppose that, unknown to us, both the candy vending machine and the slot machine are broken, and so neither one is able to dispense anything. It would not be very long before we stopped depositing coins into the broken candy machine. Probably at most we would try only two or three times before leaving the machine in disgust. But the story would be quite different with the broken slot machine. Here, we would drop in money for a considerably longer time, even though there would be no payoff.

In formal terms, we can see the difference between the two reinforcement schedules: partial reinforcement schedules (such as those provided by slot machines) maintain performance longer than do continuous reinforcement schedules (such as those established in candy vending machines) before extinction —the disappearance of the conditioned response—occurs.

Certain kinds of partial reinforcement schedules produce stronger and lengthier responding before extinction than do others. Although many dif- ferent partial reinforcement schedules have been examined, they can most readily be put into two categories: schedules that consider the number of responses made before reinforcement is given, called fixed-ratio and variable- ratio schedules, and those that consider the amount of time that elapses before reinforcement is provided, called fixed-interval and variable-interval schedules (Svartdal, 2003; Pellegrini et al., 2004; Gottlieb, 2006).

Fixed- and Variable-Ratio Schedules In a fixed-ratio schedule, reinforcement is given only after a specific number of responses. For instance, a rat might receive a food pellet every 10th time it pressed a lever; here, the ratio would be 1:10. Similarly, garment workers are generally paid on fixed-ratio schedules: they receive a specific number of dollars for every blouse they sew. Because a greater rate of production means more reinforcement, people on fixed-ratio schedules are apt to work as quickly as possible (see Figure 3 ).

In a variable-ratio schedule, reinforcement occurs after a varying num- ber of responses rather than after a fixed number. Although the specific number of responses necessary to receive reinforcement varies, the number of responses usually hovers around a specific average. A good example of a variable-ratio schedule is a telephone salesperson’s job. She might make a sale during the third, eighth, ninth, and twentieth calls without being successful during any call in between. Although the number of responses that must be made before making a sale varies, it averages out to a 20 percent success rate. Under these circumstances, you might expect that the salesperson would try to make as many calls as possible in as short a time as possible. This is the case with all variable-ratio schedules, which lead to a high rate of response and resis- tance to extinction.

Fixed- and Variable-Interval Schedules: The Passage of Time In contrast to fixed- and variable-ratio schedules, in which the crucial factor is the number of responses, fixed- interval and variable- interval schedules focus on the amount of time that has elapsed since a person or animal was rewarded.

Fixed-ratio schedule A schedule by which reinforcement is given only after a specific number of responses are made.

Variable-ratio schedule A schedule by which reinforcement occurs after a varying number of responses rather than after a fixed number.

Fixed-ratio schedule A schedule by which reinforcement is given only after a specific number of responses are made.

Variable-ratio schedule A schedule by which reinforcement occurs after a varying number of responses rather than after a fixed number.

study alert Remember that the different

schedules of reinforcement affect the rapidity with

which a response is learned and how long it

lasts after reinforcement is no longer provided.

Module 16 operant conditioning 175

fel77023_ch05_160-197.indd 175 11/26/08 10:49:37 PM

176 Chapter 5 learning

One example of a fixed-interval schedule is a weekly paycheck. For people who receive regular, weekly paychecks, it typically makes relatively little difference exactly how much they produce in a given week.

Because a fixed-interval schedule provides reinforcement for a response only if a fixed time period has elapsed, overall rates of response are rela- tively low. This is especially true in the period just after reinforcement, when the time before another reinforcement is relatively great. Students’ study habits often exemplify this reality. If the periods between exams are relatively long (meaning that the opportunity for reinforcement for good performance is given fairly infrequently), students often study minimally

or not at all until the day of the exam draws near. Just before the exam, how- ever, students begin to cram for it, signaling a rapid increase in the rate of their

Fixed-interval schedule A schedule that provides reinforcement for a response only if a fixed time period has elapsed, making overall rates of response relatively low.

Fixed-interval schedule A schedule that provides reinforcement for a response only if a fixed time period has elapsed, making overall rates of response relatively low.

Figure 3 Typical outcomes of different reinforcement schedules. (A) In a fixed- ratio schedule, short pauses occur after each response. Because the more responses, the more