Thorndike's law of effect states that

Operant Conditioning

Psychologists like B. F. Skinner have studied how we can use operant conditioning to change the behavior of people and animals. Drawing on your personal experience, choose a person or animal whose behavior you want to change. (You may select your own behavior for this question if you wish.) How could you use operant conditioning to change the behavior of this person or animal?
In a multi-paragraph essay, describe your plan to change this behavior. Be sure to mention what type of reinforcer and reinforcement schedule you would use and explain why you made those particular choices. Include information from class materials, readings, and research on operant conditioning to support your discussion.

What Is Operant Conditioning?

The discussion of behaviorism in Chapter 1 introduced you to Edward Thorndike and his law of effect. To recap, Thorndike had observed the learning that took place when a cat tried to escape one of his “puzzle boxes.” According to Thorndike, the cats learned to escape by repeating actions that produced desirable outcomes and by eliminating behaviors that produced what he called “annoying” outcomes, or outcomes featuring either no useful effects or negative effects (1913, p. 50). Consequently, the law of effect states that a behavior will be “stamped into” an organism’s repertoire depending on the consequences of the behavior (Thorndike, 1913, p. 129).

The association between a behavior and its consequences is called operant or instrumental conditioning. In this type of learning, organisms operate on their environment, and their behavior is often instrumental in producing an outcome. B. F. Skinner extended Thorndike’s findings using an apparatus that bears his name—the Skinner box, a modified cage containing levers or buttons that can be pressed or pecked by animals (see Figure 8.7).

The Skinner Box.

A specially adapted cage called a Skinner box, after behaviorist B. F. Skinner, allows researchers to investigate the results of reinforcement and punishment on the likelihood that the rat will press the bar.

Operant conditioning differs from classical conditioning along several dimensions. By definition, classical conditioning is based on an association between two stimuli, whereas operant conditioning occurs when a behavior is associated with its consequences. Classical conditioning generally works best with relatively involuntary behaviors, such as fear or salivation, whereas operant conditioning involves voluntary behaviors, like walking to class or waving to a friend.

8-4aTypes of Consequences
As we all know from experience, some types of consequences increase behaviors, while others decrease behaviors. Skinner divided consequences into four classes: positive reinforcement, negative reinforcement, positive punishment, and negative punishment. Both types of reinforcement increase their associated behaviors, whereas both types of punishment decrease associated behaviors (see Table 8.2).

Table 8.2

Types of Consequences

Add stimulus to environment

Remove stimulus from environment

Increase behavior

Positive reinforcement

Negative reinforcement

Decrease behavior

Positive punishment

Negative punishment

We all have unique sets of effective reinforcers and punishers. You might think that getting an A in a course is reinforcing, making all those extra hours spent studying worthwhile, but top grades may be less meaningful to the student sitting next to you, who came to college for the social life. A parent might spank a child, believing that spanking is an effective form of punishment, only to find that the child’s unwanted behavior is becoming more rather than less frequent. For some children, the reward of getting the parent’s attention overrides the discomfort of the spanking part of the interaction. In other words, the identity of a reinforcer or punisher is defined by its effects on behavior, not by some intrinsic quality of the consequence. The only accurate way to determine the impact of a consequence is to check your results. If you think you’re reinforcing or punishing a behavior but the frequency of the behavior is not changing in the direction you expect, try something else.

Positive Reinforcement
By definition, a positive reinforcement increases the frequency of its associated behavior by providing a desired outcome. Again, each person has a menu of effective reinforcements. In a common application of operant conditioning, children with autism spectrum disorder are taught language, with candy serving as the positive reinforcement. Benjamin Lahey tells of his experience trying to teach a child with autism spectrum disorder to say the syllable “ba” to obtain an M&M candy (Lahey, 1995). After 4 hours without progress, Lahey turned to the child’s mother in frustration, asking her what she thought might be the problem. The mother calmly replied that her son didn’t like M&Ms. Lahey switched to the child’s preferred treat, chopped carrots, and the child quickly began saying “ba.” Chopped carrots are probably not the first reinforcer you would try with a 4-year-old boy, but in this case, they made all the difference.

Thinking Scientifically

Why Do People Deliberately Injure Themselves?
Edward Thorndike’s Law of effect stipulates that behaviors followed by positive consequences are more likely to be repeated in the future, and that behaviors followed by negative consequences are less likely to be repeated. Why, then, do large numbers of people, particularly in adolescence, engage in self-injury, or deliberate physical damage without suicidal intent (Klonsky & Muehlenkamp, 2007)? Up to 25% of teens have tried self-injury at least once (Lovell & Clifford, 2016). Most initiate self-injury while in middle school (grades 6 through 8), and approximately 6% of college students continue to self-injure.

As this chapter has detailed, reward and punishment are in the eye of the beholder. The first challenge that we face in our analysis of self-injury is the assumption that pain is always a negative consequence. For most of us, it is. However, adolescents who engage in self-injury report feelings of relief or calm, despite the obvious pain that they inflict on themselves. Such feelings probably reinforce further bouts of self-injury. Self-injury often occurs in response to feelings of anger, anxiety, and frustration, and alleviation of these negative feelings might reward the injurious behavior (Klonsky, 2007; Klonsky & Muehlenkamp, 2007). Finally, injury is associated with the release of endorphins, our bodies’ natural opioids. The positive feelings associated with endorphin release also might reinforce the behavior.

Self-injury frequently occurs in people diagnosed with psychological disorders, such as depression, anxiety disorders, eating disorders, or substance abuse, which are discussed further in Chapters 7 and 14. Others engaging in the behavior have a history of sexual abuse. Observations that captive animals in zoos and laboratories are often prone to self-injury might provide additional insight into the causes of this behavior (Jones & Barraclough, 1978). Treatment usually consists of therapy for any underlying psychological disorders, along with avoidance, in which the person is encouraged to engage in behaviors that are incompatible with self-harm. To assist these individuals further, we need to be able to see reward and punishment from their perspective, not just our own.

If the consequences of a behavior influence how likely a person is to repeat the behavior in the future, how can we explain the prevalence of self-injury? Why don’t the painful consequences of the behavior make people stop? In situations like this, operant conditioning tells us that we need to look for possible reinforcers for the behavior that override the painful outcomes. In the case of self-injury, people report feeling calm and relief. To treat such behaviors effectively, psychologists need to understand what advantages they provide from the perspective of the person doing the behavior.

Rusig/Alamy Stock Photo

The Premack principle can help you maintain good time management. If you prefer socializing to studying, use the opportunity to socialize as a reward for meeting your evening’s study goals.

If everyone has a different set of effective reinforcers, how do we know which to use? A simple technique for predicting what a particular animal or person will find reinforcing is the Premack principle, which states that whatever behavior an organism spends the most time and energy doing is likely to be important to that organism (Premack, 1965). It is possible, therefore, to rank people’s free-time activities according to their priorities. If Lahey had been able to observe his young client’s eating habits before starting training, it is unlikely that he would have made the mistake of offering M&Ms as reinforcers. The opportunity to engage in a higher-priority activity is always capable of rewarding a lower-priority activity. Your grandmother may never have heard of Premack, but she knows that telling you to eat your broccoli to get an ice cream generally works.

Both Thorndike and Skinner agreed that positive reinforcement is a powerful tool for managing behavior. In our later discussion of punishment, we will argue that the effects of positive reinforcement are more powerful than the effects of punishment. Unfortunately, in Western culture, we tend to provide relatively little positive reinforcement. We are more likely to hear about our mistakes from our boss than all the things we’ve done correctly. It is possible that we feel entitled to good treatment from others, so we feel that we should not have to provide any reward for reasonably expected behaviors. The problem with this approach is that extinction occurs in operant, as well as in classical, conditioning. A behavior that is no longer reinforced drops in frequency. By ignoring other people’s desirable behaviors instead of reinforcing them, perhaps with a simple thank-you, we risk reducing their frequency.

According to the Premack principle, a preferred activity can be used to reinforce a less preferred activity. Most children prefer candy over carrots, so rewarding a child with candy for eating carrots often increases carrot consumption. One little boy with autism spectrum disorder, however, preferred carrots to M&Ms, and his training proceeded more smoothly when carrot rewards were substituted for candy rewards.

Carolyn Jenkins/Alamy Stock Photo; Itani/Alamy Stock Photo

Some reinforcers, known as primary reinforcers, are effective because of their natural roles in survival, such as food. Others must be learned. We are not born valuing money, grades, or gold medals. These are examples of conditioned reinforcers , also called secondary reinforcers, that gain their value and ability to influence behavior from being associated with other things we value. Here, we see an intersection between classical and operant conditioning. If you always say “good dog” before you provide your pet with a treat, saying “good dog” becomes a CS for food (the UCS) that can now be used to reinforce compliance with commands to come, sit, or heel (operant behaviors). Classical conditioning establishes the value of “good dog,” and operant conditioning describes the use of “good dog” to reinforce the dog’s voluntary behavior.

Serena Williams “loves” her Wimbledon trophy not for its intrinsic value (you can’t eat it, etc.), but because trophies have become conditioned reinforcers.

PA Images/Alamy Stock Photo

Many superstitious behaviors, like wearing your “lucky socks,” can be learned through operant conditioning. Operant conditioning does not require a behavior to cause a positive outcome to be strengthened. All that is required is that a behavior be followed by a positive outcome. Unless you suddenly have a string of bad performances while wearing the lucky socks, you are unlikely to have an opportunity to unlearn your superstition.

Humans are capable of generating long chains of conditioned reinforcers extending far into the future. We might ask you why you are studying this textbook right now, at this moment. A psychologist might answer that you are studying now because studying will be reinforced by a good grade at the end of the term, which in turn will be reinforced by a diploma at the end of your college education, which in turn will be reinforced by a good job after graduation, which in turn will be reinforced by a good salary, which will allow you to live in a nice house, drive a nice car, wear nice clothes, eat good food, and provide the same for your family in the coming years.

Negative Reinforcement
Negative reinforcement , which sounds contradictory, involves the removal of unpleasant consequences from a situation to increase the frequency of an associated behavior. Negative reinforcement increases the frequency of behaviors that allow an organism to avoid, turn off, or postpone an unpleasant consequence; these are sometimes called escape and avoidance behaviors.

Let’s look at a laboratory example of negative reinforcement before tackling real-world examples. If a hungry rat in a Skinner box learns that pressing a bar produces food, a positive consequence, we would expect the frequency of bar pressing to increase. This would be an instance of positive reinforcement. However, if pressing the bar turns off or delays the administration of an electric shock, we would still expect the frequency of bar pressing to increase. This would be an instance of negative reinforcement.

Be careful to avoid confusing negative reinforcement with punishment, which is covered in the next section. By definition, a punishment decreases the frequency of the behaviors that it follows, whereas both positive and negative reinforcers increase the frequency of the behaviors that they follow. Returning to our Skinner box example, the rat’s bar pressing increases following both positive reinforcement (food) and negative reinforcement (turning off a shock). If we shocked the rat every time it pressed the bar (punishment), it would stop pressing the bar quickly.

Many everyday behaviors are maintained by negative reinforcement. We buckle up in our cars to turn off annoying beeps or bells, open umbrellas to avoid getting wet, scratch an insect bite to relieve the itch, take an aspirin to escape a headache, apply sunscreen to avoid a sunburn or skin cancer, and apologize to avoid further misunderstandings with a friend.

In many real-world cases, positive and negative reinforcement act on behavior simultaneously. A heroin addict uses the drug to obtain a state of euphoria (positive reinforcer) but also to eliminate the unpleasant symptoms of withdrawal (negative reinforcer). You might study hard to achieve high grades (positive reinforcers) while also being motivated by the need to avoid low grades (negative reinforcers).

Punishment
A punishment is any consequence that reduces the frequency of an associated behavior. Positive punishment refers to applying an aversive consequence that reduces the frequency of or eliminates a behavior. As described previously, we can demonstrate that a rat will quickly stop pressing a bar if each press results in an electric shock. Negative punishment involves the removal of something desirable. In the Skinner box, we can change the rules for a rat that has learned previously to press a bar for food. Now, food is made available unless the rat presses the bar. Under these conditions, the rat will also quickly stop pressing the bar.

Putting up an umbrella to avoid getting wet from the rain is an example of a negatively reinforced behavior.

Greg McWilliams/Icon SMI 408/Newscom/Icon Sportswire/Atlanta Ga United States

Thorndike and Skinner were in agreement about the relative weakness of punishment as a means of controlling behavior. Part of the weakness of punishment effects arises from the difficulties of applying punishment effectively in real contexts. Three conditions must be met for punishment to have observable effects on behavior: significance, immediacy, and consistency (Schwartz, 1984).

As we observed with reinforcement, consequences have to matter to the person or animal receiving them (i.e., significance). If we use a punisher that is too mild for a particular individual, there is little incentive for that person to change behavior. College campuses usually charge a fairly significant amount of money for parking illegally. However, there will typically be some students for whom that particular punishment is not sufficient to ensure that they will park legally. How high would a parking fee have to be to gain complete compliance on the part of the university community? What if you risked the death penalty for parking illegally? We can be fairly certain that most people would leave their cars at home rather than risk that particular consequence. The point is that punishment can work if a sufficiently severe consequence is selected, but using the amount of force needed to produce results is rarely considered practical and ethical. Free societies have long-standing social prohibitions against cruel and unusual punishments, and these conventions are incompatible with using the force that may be needed to change the behavior of some individuals.

Immediate punishment is more effective than delayed punishment (i.e., immediacy). For the rat in the Skinner box, delays of just 10 seconds can reduce the effectiveness of electric shock as a punisher. Humans are more capable than rats at bridging long intervals (Kamin, 1959). Nonetheless, the same principle holds true. Delayed punishment is less effective than immediate punishment. We should not be too surprised that the months or years that are required to try and convict a serious criminal may greatly reduce the impact of imprisonment on that person’s subsequent behavior.

The likelihood of getting a ticket influences drivers’ behavior. At an intersection with cameras, drivers are unlikely to run a red light, but at other intersections, behavior might be determined by whether a police officer is nearby.

Ernest R. Prim/Shutterstock.com

Our final requirement for effective punishment is its uniform application (i.e., consistency). College students are a prosocial, law-abiding group as a whole, yet many confess to determining their highway speed based on the presence or absence of a police car in their rearview mirrors. The experience of exceeding the speed limit without consequence weakens the ability of the possibility of tickets and fines to influence behavior. However, at intersections known to be controlled by cameras, compliance is generally quite high. If you are certain that running a red light will result in an expensive ticket, it would be foolish to test the system.

To reduce undesirable behaviors, Skinner recommended extinction as an alternative to punishment (Skinner, 1953). In the discussion of classical conditioning earlier in this chapter,the term extinction was used to refer to the disappearance of CRs that occurs when the CS no longer signals the arrival of a UCS. Extinction in operant conditioning has a similar meaning. Learned behaviors stop when they are no longer followed by a reinforcing consequence. Attention is a powerful reinforcer for humans, and ignoring negative behavior should extinguish it. Parents and teachers cannot look the other way when one child is being physically aggressive toward another, but in many other instances, Skinner’s approach is quite successful in reducing the frequency of unwanted behaviors (Brown & Elliot, 1965). Although ignoring a child’s tantrums can be embarrassing for many parents, this can be an effective strategy for reducing their frequency.

Connecting to Research

Does Age Influence the Effects of Consequences?
In this chapter, we have learned that rewards increase and punishments decrease the frequency of associated behaviors. But how do these consequences compare to each other? Do people learn faster when rewarded or punished? Is the relative effectiveness of reward and punishment the same for people of all ages? Participants were asked to choose the “correct” item from a pair of abstract symbols, and that their overall points (11 for a correct choice and 21 for an incorrect choice) would determine a final monetary prize (Palminteri, Kilford, Coricelli, & Blakemore, 2016).

The Question: Do adolescents and adults respond differently to consequences?

Methods
In this British study, 38 people (20 adults between the ages of 18 and 32 and 18 adolescents between the ages of 12 and 17) participated. They received 5 pounds (about $6.20) for participating, plus anywhere from zero to 10 pounds ($12.40) based on their performance in the task. They viewed pairs of abstract items (the Agathodaimon alphabet). As shown in Figure 8.8, their choices were either rewarded (happy smiley and +1 point) or punished (unhappy smiley and −1 point).

Figure 8.8Research Protocol for Studying the Effects of Reward and Punishment.

Participants viewed these slides in order. First, they fixated on the cross. Next, they see both figures and select one. Finally, they receive feedback in the form of a smiley and point total (in this case, our participant was correct).

Source: S. Palminteri, E. J. Kilford, G. Coricelli, & S.-J. Blakemore (2016). “The Computational Development of Reinforcement Learning During Adolescence.” PLoS Computational Biology, 12(6), e1004953.

Ethics
Although the topic of punishment might make it sound like this study raises more than the typical ethical concerns about research, facing an unhappy smiley and failing to earn 10 pounds ($12.40) are unlikely to cause the participants significant distress. The “risks” of the study would be spelled out for prospective participants in an informed consent process. The participation of adolescents would require parent or guardian approval plus participant assent. In other words, adolescents can’t be forced to participate by their parents, but they cannot participate without parental approval.

Results
The adults learned equally well from punishment and reinforcement. In contrast, the adolescents were more likely to learn from reinforcement than from punishment. In other words, the adolescents were more likely to seek rewards than to try to avoid punishments.

Conclusions
This research demonstrated that decision making based on the probabilities of reward and punishment continues to develop through adolescence into young adulthood. Unlike the adults, who were as likely to seek rewards as they were to try to avoid punishments, the adolescents appeared to pay more attention to rewards.

Among the many implications of this study, we might conclude that using positive feedback to support learning in adolescents is likely to produce more benefits that using negative feedback. In other words, teens might learn more at school about what they’re doing correctly than what they’re doing incorrectly. College students, in contrast, are likely to benefit from both types of feedback.

8-4bSchedules of Reinforcement
Reinforcing a behavior every time it occurs is known as continuous reinforcement. Although it is highly desirable to use continuous reinforcement when a new behavior is being learned, it is inconvenient to do so forever. Dog owners want their dogs to walk with a loose leash, but once this skill is learned, the owners don’t want to carry dog treats for reinforcement whenever they walk their dogs. Once we deviate from continuous reinforcement, however, the manner in which we do so may have a dramatic impact on the target behavior. To obtain the results we want, it is helpful to understand what happens when we use partial reinforcement , or the reinforcement of the desired behavior on some occasions, but not others.

Concerns about the effects of piecework on worker well-being contributed to the Fair Labor Standards Act of 1938, which included a provision for a minimum hourly wage.

Psychologists have identified many ways to apply partial reinforcement, but we will concentrate on two variations: ratio schedules and interval schedules. In a ratio schedule of partial reinforcement, reinforcement depends on the number of times a behavior occurs. In an interval schedule of partial reinforcement, reinforcement depends on the passage of a certain amount of time. Either type of schedule, ratio or interval, can be fixed or variable. In fixed schedules, the requirements for reinforcement never vary. In variable schedules, the requirements for reinforcement are allowed to fluctuate from trial to trial, averaging a certain amount over the course of a learning session.

Fixed Ratio Schedules
A fixed ratio (FR) schedule requires that a behavior occur a set number of times for each reinforcer. Continuous reinforcement, discussed earlier, is equivalent to a FR of 1. If we now raise our requirement to two behaviors per reinforcer, we have a FR schedule of 2, and so on. Using the Skinner box, we can investigate the influence of FR schedules on the rate at which a rat will press a bar for food. To do so, we track cumulative responses as a function of time. FR schedules produce a characteristic pattern of responding. In general, responses are fairly steady, with a significant pause following each reward. As the amount of work for each reward is increased, responding becomes slower (see Figure 8.9).

Figure 8.9Schedules of Reinforcement.

The schedule used to deliver reinforcement has a big impact on the resulting behavior. In general, the variable schedules produce higher rates of responding than do their fixed counterparts. The fixed interval schedule produces a characteristic pattern of low rates of responding at the beginning of the interval and accelerated responding as the end of the interval approaches.

Diverse Voices in Psychology

Does Physical Punishment Have Different Effects in Different Cultural Contexts?
Psychologists typically recommend against the use of physical punishment with children, largely due to research showing a relationship between the use of physical punishment and increased aggressiveness on the part of a child. In addition, as you have seen in this chapter, there are many alternative ways to manage behavior successfully.

As is the case with many types of psychological research, however, the classic studies on physical punishment and child aggression were conducted with middle-class, white American families. How representative are these samples of families in general? Some researchers believe that they are not representative and that physical punishment effects depend very much on cultural context (Deater-Deckard & Dodge, 1997). These researchers believe that physical punishment in cultures in which it is considered “normal” has a much less detrimental effect than in cultures where it is considered less normal.

Racial and ethnic groups vary in the frequency with which they use physical punishment (Gershoff, Lansford, Sexton, Davis-Kean, & Sameroff, 2012). In a sample of over 11,000 U.S. families with kindergarten-aged children, rates of spanking were generally high (about 80%) across all groups, with 89% of African Americans, 79% of whites, 80% of Hispanics, and 73% of Asians reporting having spanked their child. When asked if they had spanked their child in the previous week, 40% of African Americans, 28% of Hispanics, 24% of whites, and 23% of Asians reported that they had done so.

Does spanking have different effects on children’s behavior across the racial and ethnic groups due to these different frequencies? Gershoff et al. (2012) concluded that it does not. Across all racial and ethnic groups, spanking was associated with higher levels of aggressive behaviors in the child, which in turn leads to more spanking in a “coercive cycle” of parenting.

Psychologists have wondered if spanking had different effects within different racial and ethnic contexts, but it does not. Spanking is associated with higher levels of child aggression, regardless of racial or ethnic context.

Sandro Di Carlo Darsa/PhotoAlto/Alamy Stock Photo

While many lines of research benefit from considerations of diversity, the message here is not modified by racial or ethnic identity—spanking children not only fails to decrease negative behaviors, but it actually appears to increase them.

In early industrial settings, workers were often paid by the piece, a real-world example of the use of an FR schedule. In other words, workers would be paid every time they produced a fixed number of products or parts on an assembly line. Most workers find this system less than ideal. If the equipment malfunctions, the worker cannot earn money. Lunch breaks would also then be viewed as loss of income rather than a helpful time of rest. Some examples of piecework remain today, including the work of most physicians, who get paid by the procedure, and service workers like plumbers or hairstylists, who get paid for finishing a specific task.

Variable Ratio Schedules
As in FR schedules, variable ratio (VR) schedules also involve counting the number of times that a behavior occurs. However, this time the required number of behaviors is allowed to fluctuate around some average amount.

In the Skinner box, we might set our VR schedule to 10 for a 1-hour session. This means that over the course of the session, the rat must press an average of 10 times to receive each food pellet. However, this schedule may mean that only 1 press delivers food on one trial, but 30 presses are required on the next trial. The rat is unable to predict when reinforcement is likely to occur, leading to a high, steady rate of responding in our cumulative record. We do not see the characteristic pausing observed following reinforcement in the FR schedule because the rat cannot predict when the next reward will occur.

One of the most dramatic real-world examples of the VR schedule is the programming of slot machines in casinos. Slot machines are human versions of Skinner boxes that use VR schedules. The casino sets the machine to pay off after some average number of plays, but the player doesn’t know whether a payoff will occur after one coin or thousands are inserted. You don’t have to observe the behavior of people playing slot machines long to see a demonstration of the high, steady responding that characterizes the VR schedule. The programming of slot machines can be sophisticated. Slot machines that are located in places where people are unlikely to return (airports and bus stations) pay off less frequently than those in places where people are more likely to play regularly.

Fixed Interval Schedules
Unlike ratio schedules, reinforcement in interval schedules depends on the passage of time rather than the number of responses produced. In a fixed interval (FI) schedule , the time that must pass before reinforcement becomes available following a single response is set to a certain amount. In the Skinner box, a rat’s first bar press starts a timer. Responses that occur before the timer counts down are not reinforced. As soon as the timer counts down to zero, the rat’s next bar press is reinforced, and the timer starts counting down again. In the FI schedule, the interval is the same from trial to trial. Animals and people have a good general sense of the passage of time, leading to a characteristic pattern of responding in FI situations. Reinforcement is followed by a long pause. As the end of the interval is anticipated, responding increases sharply. A graph of the number of bills passed by the U.S. Congress as a function of time looks similar to the rat’s performance on an FI schedule in the Skinner box (Weisberg & Waldrop, 1972). Few bills are passed at the beginning of a session, but many are passed at the end (see Figure 8.10).

Workers in the garment industry are often paid by the piece, or with a set amount of money for each finished garment. This compensation system is an example of a fixed ratio (FR) schedule. Because workers cannot make money when their equipment breaks down and they tend to view lunch and other breaks as costing them money, this schedule is not considered to be fair to workers.

SCPhotos/Alamy Stock Photo

Figure 8.10Congress and Fixed Interval (FI) Behavior.

As the end of a congressional session approaches, the U.S. Congress begins to pass more bills in patterns that look similar to the behavior of rats on FI schedules in Skinner boxes.

Library of Congress Prints and Photographs Division [LC-HS503-1388]

Variable Interval Schedules
As you already may have guessed, the variable interval (VI) schedule is characterized by an interval that is allowed to fluctuate around some average amount over the course of a session. This time, our bar-pressing rat experiences intervals that range around some average amount (say, 2 minutes). On one trial, the rat may obtain reinforcement after only 30 seconds, whereas the next trial may involve an interval of 5 minutes. Over the session, the average of all intervals is 2 minutes. As in the VR situation, we see a high, steady rate of responding.

You are probably quite familiar with VI schedules, in the form of pop quizzes administered by your professors. Your professor might tell you that there will be five quizzes given during the term, but the timing of the quizzes remains a surprise. You might have the first two only 1 day apart, followed by a 2-week interval before the next quiz. Your best strategy, like the rat in the Skinner box on a VI schedule, is to emit a high, steady rate of behavior.

Partial Reinforcement Effect in Extinction
Many parents have regretted the day that they unintentionally put an unwanted behavior on a partial reinforcement schedule by uttering the words, “OK, just this once.” Perhaps a parent is strongly opposed to buying candy for a child at the supermarket checkout counter (where, because of John Watson and his applications of psychology to advertising, candy is displayed conveniently at child’s-eye height). Then comes the fateful day when the parent is late coming home from work, the child is hungry because dinner is delayed, and unintentionally, the parent gives in “just this once,” putting begging for candy on a variable schedule. Subsequently, when the parent returns to the previous refusal to buy candy, a high, steady rate of begging behavior occurs before it finally extinguishes.

Most casinos feature a large number of slot machines, which are essentially Skinner boxes for people. The slot machine is programmed on a variable ratio (VR) schedule, which means that the player cannot predict how many plays it will take to win. In response, players exhibit the same high, steady rate of responding that we observe in rats working on VR schedules in the laboratory.

Tetra Images/Photoshot

In the laboratory once more, we compare the behavior of two rats in Skinner boxes. One is working on a continuous (or FR 1) schedule of reinforcement. The other is working on a partial schedule of reinforcement (perhaps a VR 3 schedule). After several sessions of training, we stop reinforcement for both rats. It may surprise you to learn that the rat working on the continuous schedule will stop pressing long before the rat accustomed to the VR 3 schedule. In other words, extinction occurs more rapidly following continuous reinforcement than following partial schedules. This outcome is known as the partial reinforcement effect in extinction .

The partial reinforcement effect probably occurs because of one of two factors, or a combination of both. First, the transition from continuous reinforcement to extinction is more obvious than the transition from a partial schedule to extinction. If you are accustomed to being paid for a babysitting job every time you work, you will definitely notice any of your employer’s failures to pay. In contrast, if your neighbor typically pays you about once a month for raking his yard each weekend, you might not notice right away that he hasn’t paid you for a while. Second, partial schedules teach organisms to persist in the face of nonreinforcement. In a sense, partial schedules teach us to work through periods in which reinforcement does not occur. Consequently, we might view extinction as just another case where continuing to perform might eventually produce reinforcement. When positive behavior is occurring, such as working on your senior thesis regularly despite a much-delayed grade, persistence is an enormous advantage. However, as shown in our earlier example of begging for candy, placing an undesirable behavior on partial reinforcement makes it more difficult to extinguish.

Fishing works according to a variable interval (VI) schedule of reinforcement. Fish (the reinforcers) are caught after periods of waiting for fish to bite that vary in length. As in laboratory demonstrations of the VI schedule, fishing usually produces a steady rate of responding.

holbox/Shutterstock.com

Comparing Schedules
What happens if you are exposed to two or more schedules of reinforcement at the same time? This scenario is realistic because we face these types of choices every day. Which is a more rewarding use of my time: studying for my midterm or making some extra money by working overtime? In making these types of choices, animals and people follow the matching law, which states that the relative frequency of responding to one alternative will match the relative reinforcement for responses on that alternative (Herrnstein & Heyman, 1979). The law powerfully accounts for the effects on behavior of frequency, magnitude, and delays in reward.

Time spent playing online video games provides an interesting example of the effects of simultaneous schedules of reinforcement. The millions of users of massively multiplayer online games, such as League of Legends and Crossfire, spend an average of 22 hours per week on their games (Yee, 2006). What compels these people to make such a lopsided choice between online interactions and real-life social experiences? One clue to this choice is that substantial numbers of players report that “the most rewarding or satisfying experience” they had over the last 7 or 30 days took place while gaming. We would assume that if the frequency and magnitude of rewards available in gaming were higher than those in real-life socializing, people would choose to spend their time accordingly.

8-4cThe Method of Successive Approximations (Shaping)
So far, this discussion of operant conditioning has centered on increasing or decreasing the frequency of a particular behavior. What happens if you want to increase the frequency of a behavior that rarely or never occurs? Most parents would like to teach their children to use good table manners, but you could wait a long time for the opportunity to reward young children for using the correct utensil to eat their food.

Fortunately, we have a method for increasing the frequency of behaviors that never or rarely occur. Using the method of successive approximations , or shaping, we begin by reinforcing spontaneous behaviors that are somewhat similar to the target behavior that we want to train. As training continues, we use gradually more stringent requirements for reinforcement until the exact behavior that we want occurs. You can think of shaping as a funnel. Using the table manners example, parents start out with generous criteria for reinforcement (“Thank you for picking up the spoon”) and gradually narrow the criteria (“Thank you for putting the spoon in the food”) until they are reinforcing only the target behavior (“Thank you for using the spoon to eat your applesauce”). One of the most positive features about the shaping process is that behavior doesn’t have to be perfect to produce reinforcement.

Whether training imaginary raptors in Jurassic World or real animals, using the method of successive approximations (shaping) can lead to the reliable performance of otherwise low-frequency behaviors. Like Chris Pratt’s character, most trainers use a combination of classical conditioning (clicker sound leads to food) and operant conditioning (approximation of desired behavior leads to clicker sound).

Pictorial Press Ltd/Alamy Stock Photo

The rats in Skinner boxes that have been described in this chapter do not spontaneously start pressing levers. We have to teach them to do so. We begin by making sure the hungry rat understands that food is available in the Skinner box. Using a remote control, we activate the food dispenser a few times. Quickly, the rat forms a classically conditioned association between the sound of the food dispenser (CS) and the arrival of food (UCS) in the cup. However, if we continue to feed the rat in this manner, it is unlikely that it will ever learn to press the bar. Indeed, there is no reason for it to do so because it already is obtaining the food it needs. So, we narrow our criteria for obtaining food from simply existing in the box to standing in the corner of the box that contains the bar. If we press our remote control every time the rat is in the correct corner, it will begin to stay there most of the time. Now we want the rat to rear on its back feet so that it is likely to hit the bar with its front feet on the way down. If we begin to reinforce the rat less frequently for staying in the corner, it will begin to explore. Eventually, the rat is likely to hit the bar with its front feet while exploring, pressing the bar. Now it will begin to press the bar on its own. In the hands of an experienced trainer, this process takes about half an hour.

Shaping involves a delicate tightrope walk between too much and too little reinforcement. If we reinforce too generously, learning stops because there is no incentive for change. If your piano teacher always tells you that your performances are perfect, you will stop trying to improve them. However, if we don’t reinforce frequently enough, the learner becomes discouraged. Reinforcement provides important feedback to the learner, so insufficient reinforcement may slow or stop the learning process.

Teaching a complex behavior requires chaining, or breaking down the behavior into manageable steps. Chaining can be done in a forward direction, such as teaching the letters of the alphabet from A to Z, or in a backward direction, such as teaching the last step in a sequence, then the next to the last, and so on. Chaining can be useful when teaching new skills, such as working independently on academic projects, to children with special needs (Pelios, MacDuff, & Axelrod, 2003). Backward chaining is used by most trainers of animals used in entertainment. For example, dogs have been taught to perform complex dances like the Macarena (Burch & Bailey, 1999). The trainer uses a verbal, gestural, or clicker cue while shaping the last step in the dance. When the dog performs this last step reliably, the trainer adds the next-to-last step, and so on until the entire complex sequence is mastered.

8-4dCognitive, Biological, and Social Influences on Operant Conditioning
Even the most radical behaviorists, including Skinner, did not deny the existence of cognitive, social, or biological influences on learning (Jensen & Burgess, 1997). Instead, behaviorists believed that internal processes followed the same rules as externally observable behavior. As Skinner (1953) wrote, “We need not suppose that events which take place within an organism’s skin have special properties…. A private event may be distinguished by its limited accessibility but not, so far as we know, by any special nature or structure” (p. 257). However, as we saw in the case of classical conditioning, the results of some operant conditioning experiments stimulated greater interest in the cognitive, social, and biological processes involved in learning.

Cognitive Influences on Operant Conditioning
One of the important principles of operant conditioning is that consequences are required in order for learning to occur. Edward Tolman (1948) challenged this notion by allowing his rats to explore mazes without food reinforcement. Subsequently, when food was placed in the goal boxes of the mazes, the previously unreinforced rats performed as well as the rats that had been reinforced all along. Tolman referred to the rats’ ability to learn in the absence of reinforcement as latent learning . He argued that the rats had learned while just exploring, but that they did not demonstrate their learning until motivated by the food reward to do so. We usually judge whether learning has occurred by observing outward behavior. Tolman’s rats remind us that there is a difference between what has been learned and what is performed. Students are all too familiar with the experience of performing poorly on exams despite having learned a great deal about the material.

In addition to challenging the role of reinforcement in learning, Tolman disputed traditional behaviorist explanations of the nature of the learning that occurred in mazes. He believed that instead of learning a simple operant “turn right for food” association, rats learned “This is where I can find food” (Tolman, 1948, 1959). After training rats to follow a path in a maze to find food, Tolman blocked the path but allowed the rats to choose from a number of additional paths. If the rats had learned a simple turn left–get food response, they should have chosen the paths that were most similar to the training path. Instead, they showed evidence of choosing paths that required them to turn in a different direction from their previously trained path, but one that led them directly to the goal (Tolman, Richie, & Kalish, 1946; see Figure 8.11).

Figure 8.11Tolman’s Maze.

Edward Tolman did not believe that rats wandering around a maze learned “turn left for food” in the way that early behaviorists believed that they did. Instead, Tolman believed that the rats were learning a more cognitive map of where they could find food. He provided evidence for his approach by blocking a learned pathway to food. If the behaviorists were right, the rats should choose the path most similar to the trained one. However, the rats did not do that. They showed evidence of having formed cognitive maps and were willing to turn in a different direction if that path led to food.

Enlarge Image

To account for his results, Tolman suggested that the rats had formed cognitive maps, or mental representations of the mazes. Map formation was viewed as a unique, nonassociative learning process that didn’t follow the previously established rules of associative learning (O’Keefe & Nadel, 1978). For example, in contrast to the gradual acquisition of learning that usually occurs in classical and operant conditioning, cognitive maps are instantly updated when new information becomes available.

Chimpanzees show considerable ability to form cognitive maps (Menzel, 1978). After being carried around a circuitous route in their one-acre compound as nine vegetables and nine fruits were placed in 18 locations, chimpanzees were released in the center of the compound. They not only navigated to each food location using the shortest pathways, but, given their preference for fruit over vegetables, they visited the spots containing fruit first. They showed no indication that they were attempting to retrace the pathway over which they were carried as the food was put in place.

Biological Influences on Operant Conditioning
Just as the work of Garcia and Koelling highlighted the need to consider biological limitations on classical conditioning, biological boundaries in operant conditioning were described by Keller and Marion Breland, two of Skinner’s former students. In their 1961 article titled “The Misbehavior of Organisms” (a wordplay on Skinner’s classic book, The Behavior of Organisms), the Brelands outlined some challenges they encountered while using operant conditioning to train animals for entertainment.

In one instance, the researchers described how they sought to train a pig to pick up large wooden coins and deposit the coins in a large, wooden “piggy bank.” Initially, all went well. The pig would quickly learn to deposit four or five coins (an example of an FR schedule) for each food reward. Eventually, however, the pig began to work slower and slower, to the point where it couldn’t obtain enough food for the day. Instead of taking the coins to the piggy bank, the pig would repeatedly toss them in the air and sniff around to find them. Raccoons trained with the coins ultimately tried to wash them instead of depositing them in the bank. The animals’ natural approach to food (the rooting by the pigs and the washing by the raccoons) began to interfere with their handling of the coins. You may already have suspected that the coins had become the object of some higher-order classical conditioning because of their relationship with food. The Brelands concluded “that these animals are trapped by strong instinctive behaviors, and clearly we have here a demonstration of the prepotency of such behavior patterns over those which have been conditioned. We have termed this phenomenon ‘instinctive drift’” (Breland & Breland, 1961, p. 683).

Keller Breland observes the performance of one of the star pupils of the I.Q. Zoo attraction he developed with his wife, Marian. Unfortunately, the Brelands discovered that the animals’ instinctive behaviors often interfered with their training. The Brelands referred to this phenomenon as “instinctive drift.”

Historical/Corbis

Social Influences on Operant Conditioning
So far, this discussion of classical and operant conditioning has focused on the individual in isolation. Learning can take place when people or animals are alone, but it often occurs in the presence of others, especially in a species as social as ours. As we will see in a later section, people are particularly likely to learn by observing others. What do we know about the impact of others on our operant learning?

The presence of others may not just promote learning; it also may be necessary for learning. Human infants learn more about language when they are listening to another person face to face than when they are watching a person speak on television (Kuhl, 2007; Meltzoff, Kuhl, Movellan, & Sejnowski, 2009). Although operant conditioning alone cannot account for language learning, as discussed in Chapters 10 and 11, these results emphasize the importance of social interaction in producing the arousal, focus, and motivation that contribute to effective learning.

Experienced whale trainer Dawn Brancheau was killed by one of her favorite killer whales during a 2010 show at Sea World in Orlando, Florida. Animal experts believed that the whale had simply reverted to normal whale behavior, similar to the instinctive drift observed by Keller and Marian Breland.

Orlando Sentinel/Getty Images

As mentioned previously in the discussion of cognitive maps, learning and the performance of learned behavior are not always identical. Our performance of learned behaviors varies depending on an interaction between the presence of others and the complexity of the learned task. For simple tasks, like pedaling a bicycle or reeling in a fishing line, the presence of others makes us perform faster, a phenomenon known as social facilitation (Triplett, 1898; also see Chapter 13). In complex tasks, such as taking a difficult college entrance exam, the presence of others can make our performance slower and poorer. Again, this effect is not restricted to complex organisms like ourselves; the same results can be observed in the lowly cockroach (Zajonc, 1965). In a straight maze leading to food, cockroaches with an audience of other cockroaches ran quickly. In a more complex maze involving several turns, the cockroaches responded to an audience by running more slowly.

8-4eApplying Operant Conditioning
Important applications of operant conditioning may be found in contemporary approaches to psychotherapy, education, advertising, politics, and many other domains.

Possibly one of the oddest applications of operant conditioning was Skinner’s secret World War II defense project, code-named Project Pigeon. Lagging well behind the Nazis in the area of guided missile technology, the United States invested $25,000 (worth about $400,000 in today’s dollars) in Skinner’s “organic homing device” (Capshew, 1993). Skinner, who had considerable experience training pigeons to peck at visual stimuli in his laboratory, now trained them to peck at a projected image of a missile’s target. Riding in a chamber within the missile, the pigeon’s pecks would be translated into updated commands for correcting the path of the bomb. Unfortunately for Skinner (but fortunately for his pigeons), Project Pigeon elicited laughter from military officers instead of approval (Skinner, 1960). Although never implemented, Project Pigeon stimulated Skinner and his intellectual descendents to look outside the laboratory for useful extensions of their work on learning.

B. F. Skinner’s Project Pigeon was one of the more bizarre applications of operant conditioning research. Pigeons enclosed in this capsule were trained to peck at projected images of bomb targets. Even though Skinner’s device was superior to other World War II missile guidance systems, it was never implemented.

Enlarge Image

Kenneth E. Behring Center, Division of Medicine & Science, National Museum of American History, Smithsonian Institution.

Token Economies
A widely used application of operant learning is the token economy . Money, in the form of coins, bills, bitcoins, or bank statements, is fairly useless. You can’t eat it, wear it, or shelter in it. Nonetheless, people value it because it takes on secondary reinforcing qualities due to its history of association with other things that have intrinsic value. The use of money to buy things of personal value is an example of a token economy. You earn money for doing certain things, and then you have the opportunity to trade the money you earned for items of value to you. This system meets the best practices criteria that have been described in the context of positive reinforcement. Each person can obtain reinforcement that has unique personal value. One friend may spend discretionary money on going out to dinner, while another invests in the stock market. Both find money reinforcing for doing work.

Token economies can be effective ways of managing behavior. Tokens, including money, can be traded for a valued reinforcement of the worker’s choice. This woman’s purchase might motivate her work, but another worker might use the same paycheck to buy a motorcycle or go on vacation.

OJO Images/Photoshot

An informed approach to compensating employees should include consideration of learning principles. “Menu” approaches to employee benefits provide an excellent example of this application. Historically, employers offered a set program of health, retirement, and other benefits to their entire workforce regardless of individual needs. We would expect this approach to be minimally reinforcing because it does not match reinforcers to worker priorities. Catering a benefits package to individual needs is more sensible. A young worker in good health might be more motivated by a benefits package that includes childcare, while a more mature worker may worry about long-term care in the event of a disability. By allowing workers to select their benefits from a menu, everybody can find something worth earning.

All of us respond positively to token economies, but they are especially useful in educational and institutional settings. Teachers provide frequent rewards in the form of checks, stars, or tickets that can be exchanged later for popcorn parties or a night without homework. The key to an effective token economy is to offer ultimate rewards that are truly valuable to the people you wish to motivate. If students don’t care about popcorn parties, offering them will have little effect. Token economies are equally useful in prison settings and in institutions serving people with an intellectual disability or mental illness.

Behavior Therapies
As we will discuss in Chapter 15, learning theories also have been applied successfully to the clinical setting in the form of behavior therapies, also called applied behavior analysis. After all, our formal definition of learning states that it involves a change in behavior, and changing behavior is precisely what therapists usually seek to do. In addition to the extinction and counterconditioning applications of classical conditioning, behavior therapies use operant conditioning concepts such as extinction, reward, and on rare occasions, punishment.

An important application of operant conditioning principles is their use in behavior therapies for conditions like autism spectrum disorder. Operant conditioning can be used to increase the frequency of language use and socially appropriate behaviors, like eye contact.

iStock.com/ktaylorg

Coupled with cognitive methods designed to address the way that people think about their circumstances, these methods comprise the most popular and effective means for treating many types of disorder, from substance abuse to depression. One of the most dramatic applications of behavior therapy is the treatment for autism spectrum disorder pioneered by O. Ivar Lovaas (Lovaas, 1996; Lovaas et al., 1966). Autism spectrum disorder is characterized by language and social deficits. Although behavior therapy doesn’t cure autism spectrum disorder, behavioral interventions, like the use of chaining described previously, typically improve an individual’s level of functioning.

Experiencing Psychology

How Do I Break a Bad Habit?
We all have behaviors that could use some improvement. Maybe we eat poorly, drink too much, smoke, or lash out angrily at others. An understanding of the processes of learning provides us with powerful tools for changing behavior. Let’s assume that your eating habits, like those of many students, do not meet the “My mom would approve” standard. Yet you are learning in your psychology course that good health habits are essential tools for managing stress (see Chapter 16). How do you bring about the necessary changes?

Before doing anything to produce change, you need to understand your current behavior. Many people have a poor understanding of what they actually eat during a day, so you could start by keeping a diary. What foods, and how much of them, do you eat? What else is going on when you eat well or poorly? What possible reinforcers or punishers are influencing your eating patterns? For example, let’s say that you observe a tendency to eat high-calorie snacks late at night while studying, even when you are not hungry. Your goal, then, is to eliminate these late-night snacks. Your baseline shows that your snacking is a social behavior. You consume these extra foods only when studying with a group. The social camaraderie and good taste of the food serve as powerful reinforcers for the behavior.

Now that you have a better understanding of your problem behavior, you are in a good position to construct a plan. An important part of your plan is to choose appropriate consequences for your behavior. Again, it is essential that we design consequences that are meaningful to individuals. As we have argued in this chapter, positive reinforcement has many advantages over punishment. You might try placing the money you’re saving on junk food in a designated jar to buy a special (nonfood) treat at the end of a successful week, or allow yourself an extra study break each night that you meet your goals. If you are convinced that the only way that you will change is through punishment, you could take an alternative approach. Although some people have successfully stopped smoking through the use of positive punishment, including electric shock (Law & Tang, 1995), this sounds quite unpleasant. Instead, a negative punishment, such as losing texting privileges for the day following a failure to meet one’s goals, might work.

iStock.com/catherine_jones

As you implement your program, track your progress and make any modifications that seem necessary. In addition to the improvement of your target behavior, a beneficial side effect of applying learning methods to your behavior is the knowledge that given the right tools, you can control your behavior.