Your search

In a discrete-trials procedure with pigeons, a response on a green key led to a 4-s delay (during which green houselights were lit) and then a reinforcer might or might not be delivered. A response on a red key led to a delay of adjustable duration (during which red houselights were lit) and then a certain reinforcer. The delay was adjusted so as to estimate an indifference point-a duration for which the two alternatives were equally preferred. Once the green key was chosen, a subject had to continue to respond on the green key until a reinforcer was delivered. Each response on the green key, plus the 4-s delay that followed every response, was called one “link” of the green-key schedule. Subjects showed much greater preference for the green key when the number of links before reinforcement was variable (averaging four) than when it was fixed (always exactly four). These findings are consistent with the view that probabilistic reinforcers are analogous to reinforcers delivered after variable delays. When successive links were separated by 4-s or 8-s “interlink intervals” with white houselights, preference for the probabilistic alternative decreased somewhat for 2 subjects but was unaffected for the other 2 subjects. When the interlink intervals had the same green houselights that were present during the 4-s delays, preference for the green key decreased substantially for all subjects. These results provided mixed support for the view that preference for a probabilistic reinforcer is inversely related to the duration of conditioned reinforcers that precede the delivery of food.

Four hundred and fifty participants were recruited from Amazon Mechanical Turk across 3 experiments to test the predictions of a hyperbolic discounting equation in accounting for human choices involving variable delays or multiple rewards (Mazur, 1984, 1986). In Experiment 1, participants made hypothetical choices between 2 monetary alternatives, 1 consisting of a fixed delay and another consisting of 2 delays of equal probability (i.e., a variable-delay procedure). In Experiment 2, participants made hypothetical monetary choices between a single, immediate reward and 2 rewards, 1 immediate and 1 delayed (i.e., a double-reward procedure). Experiment 3 also used a double-reward procedure, but with 2 delayed rewards. Participants in all 3 experiments also completed a standard delay-discounting task. Finally, 3 reward amounts were tested in each type of task ($100, $1000, and $5000). In the double-reward conditions (Experiments 2 and 3), the results were in good qualitative and quantitative agreement with Mazur's model (1984, 1986). In contrast, when participants made choices involving variable delays (Experiment 1), there was relatively poor qualitative and quantitative agreement with this model. These results, along with our previous findings, suggest the structure of questions in hypothetical tasks with humans can be a strong determinant of the choice pattern.

Twelve pigeons responded on concurrent variable-interval schedules that delivered token stimuli (stimulus lights for some pigeons, and white circles on the response keys for others). During exchange periods, each token could be exchanged for food on a fixed-ratio 1 schedule. Across conditions, the exchange requirements ( number of tokens that had to be earned before they could be exchanged for food) varied between one and four for the two response keys. The main findings were that the pigeons' response percentages varied as a function of the number of tokens earned at any given moment, and they were determined by both the delays to food and by the number of food deliveries in the exchange periods. In some conditions, tokens had to be earned but were not visible during the variable-interval schedules for one or both keys. When one key had visible tokens and the other did not, the pigeons showed a preference for the key without visible tokens. A model based on the matching law and a hyperbolic delay-discounting equation could account for the main patterns of choice responding, and for how response percentages changed as successive tokens were earned. The results are consistent with the view that the token stimuli served as discriminative stimuli that signaled the current delays to food.

Prior research has shown that nonhumans show an extreme preference for variable-over fixed-delays to reinforcement. This well-established preference for variability occurs because a reinforcer's strength or “value” decreases according to a curvilinear function as its delay increases. The purpose of the present experiments was to investigate whether this preference for variability occurs with human participants making hypothetical choices. In three experiments, participants recruited from Amazon Mechanical Turk made choices between variable and fixed monetary rewards. In a variable-delay procedure, participants repeatedly chose between a reward delivered either immediately or after a delay (with equal probability) and a reward after a fixed delay (Experiments 1 and 2). In a double-reward procedure, participants made choices between an alternative consisting of two rewards, one delivered immediately and one after a delay, and a second alternative consisting of a single reward delivered after a delay (Experiments 1 and 3). Finally, all participants completed a standard delay-discounting task. Although we observed both curvilinear discounting and magnitude effects in the standard discounting task, we found no consistent evidence of a preference for variability-as predicted by two prominent models of curvilinear discounting (i.e., a simple hyperbola and a hyperboloid)-in our variable-delay and double-reward procedures. This failure to observe a preference for variability may be attributed to the hypothetical, rule-governed nature of choices in the present study. In such contexts, participants may adopt relatively simple strategies for making more complex choices.

Parallel experiments with rats and pigeons examined reasons for previous findings that in choices with probabilistic delayed reinforcers, rats' choices were affected by the time between trials whereas pigeons' choices were not. In both experiments, the animals chose between a standard alternative and an adjusting alternative. A choice of the standard alternative led to a short delay (1 s or 3 s), and then food might or might not be delivered. If food was not delivered, there was an "interlink interval," and then the animal was forced to continue to select the standard alternative until food was delivered. A choice of the adjusting alternative always led to food after a delay that was systematically increased and decreased over trials to estimate an indifference point--a delay at which the two alternatives were chosen about equally often. Under these conditions, the indifference points for both rats and pigeons increased as the interlink interval increased from 0 s to 20 s, indicating decreased preference for the probabilistic reinforcer with longer time between trials. The indifference points from both rats and pigeons were well described by the hyperbolic-decay model. In the last phase of each experiment, the animals were not forced to continue selecting the standard alternative if food was not delivered. Under these conditions, rats' choices were affected by the time between trials whereas pigeons' choices were not, replicating results of previous studies. The differences between the behavior of rats and pigeons appears to be the result of procedural details, not a fundamental difference in how these two species make choices with probabilistic delayed reinforcers.

An adjusting-delay procedure was used to study the choices of pigeons and rats when both delay and amount of reinforcement were varied. In different conditions, the choice alternatives included one versus two reinforcers, one versus three reinforcers, and three versus two reinforcers. The delay to one alternative (the standard alternative) was kept constant in a condition, and the delay to the other (the adjusting alternative) was increased or decreased many times a session so as to estimate an indifference point--a delay at which the two alternatives were chosen about equally often. Indifference functions were constructed by plotting the adjusting delay as a function of the standard delay for each pair of reinforcer amounts. The experiments were designed to test the prediction of a hyperbolic decay equation that the slopes of the indifference functions should increase as the ratio of the two reinforcer amounts increased. Consistent with the hyperbolic equation, the slopes of the indifference functions depended on the ratios of the two reinforcer amounts for both pigeons and rats. These results were not compatible with an exponential decay equation, which predicts slopes of 1 regardless of the reinforcer amounts. Combined with other data, these findings provide further evidence that delay discounting is well described by a hyperbolic equation for both species, but not by an exponential equation. Quantitative differences in the y-intercepts of the indifference functions from the two species suggested that the rate at which reinforcer strength decreases with increasing delay may be four or five times slower for rats than for pigeons.

Twenty acquisition curves were obtained from each of 8 pigeons in a free-operant choice procedure. Every condition began with a phase in which two response keys had equal probabilities of reinforcement, and, as a result, subjects' responses were divided fairly evenly between the two keys. This was followed by a phase in which one key had a higher probability of reinforcement than the other, and the development of preference was observed. In all but a few cases, response proportions increased for the key with the higher probability of reinforcement. In most conditions, the two probabilities differed by .06, but the actual probabilities varied (from .16 and .10 in one condition to .07 and .01 in another). Development of preference for the key with the higher probability of reinforcement was slower when the ratio of the two reinforcement probabilities was small (.16/.10) than when it was large (.07/.01). This finding is inconsistent with the predictions of several different quantitative models of acquisition, including the kinetic model (Myerson & Miezin, 1980) and the ratio-invariance model (Horner & Staddon, 1987). However, the finding is consistent with a hypothesis based on Weber's law, which states that the two alternatives are more discriminable when the ratio of their reinforcement probabilities is larger, and, as a result, the acquisition of preference is faster.

This experiment measured pigeons' choices between delayed reinforcers and fixed‐ratio schedules in which a force of approximately 0.48 N was needed to operate the response key. In ratio‐delay conditions, subjects chose between a fixed‐ratio schedule and an adjusting delay. The delay was increased or decreased several times a session in order to estimate an indifference point—a delay duration at which the two alternatives were chosen about equally often. Each ratio‐delay condition was followed by a delay‐delay condition in which subjects chose between the adjusting delay and a variable‐time schedule, with the components of this schedule selected to match the ratio completion times of the preceding ratio‐delay condition. The adjusting delays at the indifference point were longer when the alternative was a fixed‐ratio schedule than when it was a matched variable‐time schedule, which indicated a preference for the matched variable‐time schedules over the fixed‐ratio schedules. This preference increased in a nonlinear manner with increasing ratio size. This nonlinearity was inconsistent with a theory that states that indifference points for both time and ratio schedules can be predicted by multiplying the choice response‐reinforcer intervals of the two types of schedules by different multiplicative constants. Two other theories, which predict nonlinear increases in preference for the matched variable‐time schedules, are discussed. 1990 Society for the Experimental Analysis of Behavior

Ten acquisition curves were obtained from each of 4 pigeons in a two‐choice discrete‐trial procedure. In each of these 10 conditions, the two response keys initially had equal probabilities of reinforcement, and subjects' choice responses were about equally divided between the two keys. Then the reinforcement probabilities were changed so that one key had a higher probability of reinforcement (the left key in half of the conditions and the right key in the other half), and in nearly every case the subjects developed a preference for this key. The rate of acquisition of preference for this key was faster when the ratio of the two reinforcement probabilities was higher. For instance, acquisition of preference was faster in conditions with reinforcement probabilities of .12 and .02 than in conditions with reinforcement probabilities of .40 and .30, even though the pairs of probabilities differed by .10 in both cases. These results were used to evaluate the predictions of some theories of transitional behavior in choice situations. A trial‐by‐trial analysis of individual responses and reinforcers suggested that reinforcement had both short‐term and long‐term effects on choice. The short‐term effect was an increased probability of returning to the same key on the one or two trials following a reinforcer. The long‐term effect was a gradual increase in the proportion of responses on the key with the higher probability of reinforcement, an increase that usually continued for several hundred trials. 1990 Society for the Experimental Analysis of Behavior

Choice responding refers to the manner in which individuals allocate their time or responding among available response options. In this article, we first review basic investigations that have identified and examined variables that influence choice responding, such as response effort and reinforcement rate, immediacy, and quality. We then describe recent bridge and applied studies that illustrate how the results of basic research on choice responding can help to account for human behavior in natural environments and improve clinical assessments and interventions.

Tversky and Kahneman (1981) told participants to imagine they were at a store about to purchase an item. They were asked if they would be willing to drive 20 min to another store to receive a $5 discount on the item's price. Most participants were willing, but only when the original price of the item was small ($15); when the original price was relatively large ($125), most said they would not drive 20 min for a $5 discount. We examined this framing effect in 296 participants, but instead used a psychophysical-adjustment procedure to obtain quantitative estimates of the discount required with different (a) item prices, (b) delays until the item's receipt, and (c) opportunity costs (in “driving” vs. “delivery” tasks). We systematically replicated Tversky and Kahneman's results, but also extended them by showing a substantial influence of opportunity costs on the consumer discounts required. A behavioral model of delay discounting—additive-utility theory—accounted for 97% of the variance in these consumer discounts.

The choice responses of four pigeons were examined in 20 periods of transition in a concurrent-chain procedure with variable-interval schedules as initial links and fixed delays to reinforcement as terminal links. In some conditions, the delays to reinforcement were different for the two terminal links, and changes in preference were recorded after the delays for the two response keys were switched. In other conditions, the reinforcer delays were equal for the two keys, but which key delivered 80% of the reinforcers was periodically switched. Choice proportions changed more quickly after a switch in reinforcement percentages chan after a switch in the delays, thereby contradicting the hypothesis that faster changes would occur when the switch in conditions was easier to discriminate. Analyses of response sequences showed that the effects: of individual reinforcers were larger and lasted longer in conditions with changing reinforcement percentages than in conditions with changing terminal-link delays. Rates of change in choice behavior do not appear to be limited by the unpredictability of variable reinforcement schedules, because the changes in behavior were slow and gradual even when there was a large and sudden change in reinforcer delays. (C) 2001 Elsevier Science B.V. All rights: reserved.

For several decades, choice has been the focus of considerable research by those who study operant behavior. This is not surprising, because the topics of choice and operant behavior are intimately intertwined. In everyday life, people can choose among a large, almost infinite set of operant behaviors, and they can choose not only which behaviors to perform, but under what conditions, at what rate, and for how long. Because choice is an essential part of human (and animal) life, it has been studied with great interest not only by behavioral psychologists, but also by decision theorists, economists, political scientists, biologists, and others. The research methods used in these different disciplines vary widely, and a review of all of the different methods for studying choice is well beyond the scope and purpose of this chapter. Instead, the chapter will focus on the techniques most frequently used in operant research—techniques that involve single-subject designs, that allow precise control of the reinforcement contingencies, and that produce (in most cases) large and clear effects on each subject’s behavior.