A consequence that brings about an increase in the frequency of a behavior is a

3.1 Operant Responses

In operant behavior, responses can accrue or remove stimuli, and the stimuli can be appetitive or aversive. In positive reinforcement and negative reinforcement response rate increases by addition of appetitive stimuli or removal of aversive stimuli; positive punishment and negative punishment decreases response rate by addition of aversive stimuli or removal of appetitive stimuli. Negative reinforcement includes escape, removal of an already present aversive stimulus, and avoidance, prevention or postponement of an aversive stimulus.

Operant responses can include everything from a rat's lever press maintained by food, an infant's crying maintained by maternal attention, and small talk maintained by social companionship. The form of an operant response depends on (a) environmental constraints, such as the height of a reinforcement lever and force required to operate it, (b) the behavior emitted by the organism, and (c) behavior elicited by the consequence (illustrated by Fig. 6). Once a response is reinforced, its form tends to be repeated in a stereotyped manner. Reinforcement will ‘capture’ whatever behavior the organism emits that is contiguous with the reinforcer; even if some features are unessential to the reinforcement contingency.

Reinforcement narrows the range of variability in emitted behavior by selecting its successful forms (not necessarily the most efficient behavior), making them more frequent and thereby displacing less effective forms. Predatory species, for example, show adjustments in hunting behavior as a result of their effectiveness. The differential reinforcement of some responses to the exclusion of others is called shaping. Shaping is analogous to natural selection in that different responses function like the phenotypic variations required for evolutionary change. Without variation, changes in reinforcement contingencies could not select new forms of behavior. Despite the tendency for reinforcement to produce stereotyped behavior, it never reduces variations in behavior entirely. Variability returns to emitted behavior in extinction, a potentially adaptive reaction to dwindling resources. Creativity can also serve as the basis for reinforcement (e.g., if reinforcers are contingent on the emission of novel behavior.)

Restriction operations, such as food deprivation, are necessary for reinforcement—the response elicited by the reinforcer, such as feeding, must have some probability of occurring for the reinforcer to be effective. In rats, restricting an activity such as running in a wheel will make the activity a reinforcer for drinking, even thought the rat is not water restricted. Findings like this suggest that reinforcers may best be understood in terms of the activities they produce and that the reinforcement process involves the organism's behaving in order to maintain set levels of activities.

Once an organism learns a response-reinforcer contingency, the likelihood of the response will reflect the value of the reinforcer. For example, if a rat drinks sugar water and becomes ill, it will no longer emit behavior previously reinforced with sugar water.

A great deal of behavior is maintained by negative reinforcement, including avoidance in which responses prevent the occurrence of aversive stimuli, and escape in which responses terminate an existing aversive stimulus. Much research has been directed to avoidance because it is maintained in the absence of contiguity—responses prevent reinforcement. In some cases, avoidance is maintained by deletion of an immediately present conditioned aversive (warning) stimulus, an avoidance condition that preserves contiguous reinforcement. But organisms will work to prevent or delay aversive events without the benefit of warning stimuli, showing that avoidance does not require a Pavlovian aversive contingency. Avoidance is closely linked to punishment because organisms frequently learn to avoid places associated with punishment, a pragmatic concern for advocates of punishment.

The timing, rate, and persistence of an operant response depends on its reinforcement schedule, a constraint on earning reinforcers that requires the organism to wait for the next available reinforcer, emit a number of responses, or some combination of both. Fig. 7 shows typical patterns of behavior on four simple schedules. In an interval schedule, a reinforcer is produced by the first response after the passage of a specified interval of time. In a ratio schedule, a reinforcer is produced after the emission of a specified number of responses. Time or response requirements can either be a fixed or variable. Response rate on both ratio and interval schedules increases with reinforcement rate up to a point, and then decreases again at high rates of reinforcement. At low rates of reinforcement or during extinction, behavior on ratio schedules alternates between bouts of rapid responding and pauses, while responses on interval schedules simply occur with lower frequency. Extinction is prolonged after exposure to reinforcement schedules—especially after exposure to long interval- or large ratio-schedules.

When rate of reinforcement in interval and ratio schedules is equated, organisms respond at higher rates on ratio schedules (although the difference is small at very high rates of reinforcement). Organisms are sensitive to the fact that the rate of reinforcement in ratio schedules increases directly as a function of response rate, but not in interval schedules.

Characteristic pausing or waiting occurs after earning a reinforcer. Waiting is determined by the currently expected schedule and not because of the reinforcer. Fixed-interval and fixed-ratio schedules produce long wait times, and variable-interval and variable-ratio schedules produce short wait times. Waiting on fixed-interval schedules is reliably a fixed proportion of the interval value. In VI and VR schedules, wait times are heavily influenced by the shortest inter-reinforcement intervals or ratios in a schedule. Waiting is obligatory—organisms will wait even when doing so reduces the immediacy and overall rate of reinforcement. In schedules that require a single response, with the reinforcer occurring a fixed time after the response, the optimal strategy is to respond immediately in order to minimize delay, but organisms still wait a time proportional to the fixed time before making the response that leads to reinforcement.

Schedules can be combined in almost limitless ways to study questions dealing with choice, conditioned reinforcement, and other complex behavioral processes. In experiments on choice between two concurrent variable-interval schedules, the relative preference of many organisms, including humans, will closely match the percentage of reinforcement provided by each schedule. With choices between different fixed-interval schedules, the preference for the shorter interval is far greater than predicted by the relative intervals, indicating that the value of a delayed reinforcer decreases according to a decelerating function over time. Conditioned reinforcers have been studied with behavior chains in which one signaled schedule is made a consequence of another, for example, a one-minute fixed interval schedule signaled by a red stimulus leads to another one-minute fixed interval signaled by a green stimulus, until a terminal reinforcer is obtained. Behavior extinguishes in the early schedules of chains with three or more schedules, to the extent that reinforcement is severely reduced. Possibly conditioned reinforcers must be contiguous with primary reinforcers to effectively maintain behavior; alternatively obligatory waiting in the early schedules extends the time to the terminal reinforcer, lengthening the time to reinforcement, and generating even longer waiting.

1.2 Generalization

When an operant behavior has been reinforced in the presence of a particular SD, the same behavior may still be emitted in the presence of similar but not identical stimulus conditions. When an operant is emitted in the presence of a stimulus similar to the original SD, this is referred to as stimulus generalization. For example, if one learns to answer the door when a doorbell of a particular sound rings, one will likely answer a door when a doorbell of a somewhat different ring occurs, even if one has never heard that particular doorbell sound before. This is an example of stimulus generalization. Strictly speaking, generalization occurs when some actual physical property of the original SD is shared with another stimulus in the presence of which a response is emitted and reinforced. A doorbell's sound may be similarly pitched and thus the operant (opening the door) may be emitted, even though the behavior has never been reinforced in the presence of that exact tone before. However, for the phenomenon to be considered generalization, there must be some formal property of the stimuli that are common. In the doorbell example, the formal property was some quality of the sound.

3 The Evolutionary Analogy

Skinner captured the essence of operant behavior in the formula ‘control of behavior by its consequences,’ and very early he pointed to the analogy between the selection of the response by the subsequent event and the mechanism at work in biological evolution. An increasingly large part of his theoretical contributions were eventually devoted to elaborating the evolutionary analogy (Skinner 1987). The generalization of the selectionist model to behavior acquisition at the individual level, initially little more than a metaphoric figure, has recently gained credentials with the theses of neurobiologists, such as Changeux's Generalised Darwinism (1983) or Edelman's Neural Darwinism (1987), who both have substantiated in ontogeny selective processes previously reserved to phylogeny. One of the main tenets of Skinner's theory converges with contemporary views in neurosciences.

Skinner extended the selectionist explanation to cultural practices and achievements, joining some schools of thought in cultural anthropology and in the history of science, such as Karl Popper's selectionist account of scientific hypotheses.

1 The Operant–Respondent Distinction

The neurochemical mechanisms that mediate reinforced or operant behavior may differ in a fundamental way from those underlying reflexes or respondent behavior. This is because environmental stimuli appear to control the two classes of behavior in fundamentally different ways. In reflexes, whether conditioned or unconditioned, the controlling stimulus precedes the response and elicits it. In operant conditioning, the controlling stimulus follows the response and elevates its subsequent probability. When the controlling stimulus precedes the response, information flow in the brain is afferent to efferent, as in the conventional reflex arc. On the other hand, when the controlling stimulus follows the response, as in reinforced behavior, the underlying brain organization seems to require an unconventional circuitry in which efferents are activated before afferents. However, the mechanisms for reinforced behavior do not require circuits that directly link efferent to afferent elements. This is because operant behaviors do not directly activate the goal-detecting afferent systems. Rather, the correct response operates on the environment to produce the goal object and it is this environmental change that activates the goal-detecting systems. Thus, although the reinforcement mechanism does not require efferent-to-afferent circuitry, it must recognize efferent–afferent contingencies and must be operated selectively by them, i.e., it must cause behavioral reinforcement only when the neuronal substrates of the correct response and goal object, in that order, are activated sequentially.

2.2 Operant Conditioning

In contrast to classical conditioning, which maintains that behaviors can be elicited by preceding conditioned stimuli, operant learning principles hold that behaviors are emitted from within, in response to the environmental stimuli that follow them. Operants themselves consist of actions that are performed on the environment that produce some consequence. Operant behaviors that bring about reinforcing environmental changes (i.e., if they provide some reward to the individual or eliminate an aversive stimuli) are likely to be repeated. In the absence of reinforcement, operants are weakened. Removing consequences (ignoring) can decrease or completely eliminate many annoying child behaviors such as whining.

B. F. Skinner, an experimental psychologist considered to be the primary proponent of operant learning theory, distinguished between two important learning processes: reinforcement (both positive and negative) and punishment. Positive reinforcement is the process by which a stimulus or event, occurring after a behavior, increases the future occurrence of a behavior. Negative reinforcement also results in an increase in the frequency of a behavior, but through a process of contingently removing an aversive stimulus following a behavior. Punishment refers to the introduction of an aversive stimulus, or removal of a positive one, following a response, resulting in a decrease in the future probability of that response. Skinner also observed that extinction occurs when the absence of any reinforcement results in a decrease or a reduction in response frequency.