Your search

In Experiment 1, pigeons' pecks on a green key led to a 5-s delay with green houselights, and then food was delivered on 20% (or, in other conditions, 50%) of the trials. Pecks on a red key led to an adjusting delay with red houselights, and then food was delivered on every trial. The adjusting delay was used to estimate indifference points: delays at which the two alternatives were chosen about equally often. Varying the presence or absence of green houselights during the delays that preceded possible food deliveries had large effects on choice. In contrast, varying the presence of the gr een or red houselights in the intertrial intervals had no effects on choice. In Experiment 2, pecks on the green key led to delays of either 5 s or 30 s with green houselights, and then food was delivered on 20% of the trials. Varying the duration of the green houselights on nonreinforced trials had no effect on choice. The results suggest that the green houselights served as a conditioned reinforcer at some rimes but not at others, depending on whether or nor there was a possibility that a primary reinforcer might be delivered. Given this interpretation of what constitutes a conditioned reinforcer, most of the results were consistent with the view that the strength of a conditioned reinforcer is inversely related to its duration.

Pigeons pecked on two response keys that delivered reinforcers on a variable-interval schedule. The proportion of reinforcers delivered by one key was constant for a few sessions and then changed, and subjects' choice responses were recorded during these periods of transition. In Experiment 1, response proportions approached a new asymptote slightly more slowly when the switch in reinforcement proportions was more extreme. In Experiment 2, slightly faster transitions were found with higher overall rates of reinforcement. The results from the first session, after a switch in the reinforcement proportions, were generally consistent with a mathematical model that assumes that the strength of each response is increased by reinforcement and decreased by nonreinforcement. However, neither this model nor other similar models; predicted the `'spontaneous recovery” observed in later sessions: At the start of these sessions, response proportions reverted toward their preswitch levels. Computer simulations could mimic the spontaneous recovery by assuming that subjects store separate representations of response strength for each session, which are averaged at the start of each new session.

In three experiments, pigeons chose between alternatives that required the completion of a small ratio schedule early in the trial or a larger ratio schedule later in the trial. Completion of the ratio requirement did not lead to an immediate reinforcer, but simply allowed the events of the trial to continue. In Experiment 1, the ratio requirements interrupted periods in which food was delivered on a variable-time schedule. In Experiments 2 and 3, each ratio requirement was preceded and followed by a delay, and only one reinforcer was delivered, at the end of each trial. Two of the experiments used an adjusting-ratio procedure in which the ratio requirement was increased and decreased over trials so as to estimate an indifference point-a ratio size at which the two alternatives were chosen about equally often. These experiments found clear evidence for `'procrastination”-the choice of a larger but more delayed response requirement. In some cases, subjects chose the more delayed ratio schedule even when it was larger than the more immediate alternative by a factor of four or more. The results suggest that as the delay to the start of a ratio requirement is increased, it has progressively less effect on choice behavior, in much the same way that delaying a positive reinforcer reduces it effect on choice.

Pigeons were presented with a concurrent-chains schedule in which terminal-link entries were assigned to two response keys on a percentage basis. The terminal links were fixed delays that sometimes ended with food and sometimes did not. In most conditions, 80% of the terminal links were assigned to one key, but a smaller percentage of the terminal links ended with food for this key, so the number of food reinforcers delivered by the two alternatives was equal. When the same terminal-link stimuli (orange houselights) were used for both alternatives, the pigeons showed a preference for whichever alternative delivered more frequent terminal links. When different terminal-link stimuli (green vs. red houselights) were used for the two alternatives, the pigeons showed a preference for whichever alternative delivered fewer terminal links when terminal-link durations were long, and no systematic preferences when terminal-link durations were short. This pattern of results was consistent with the predictions of Grace's (1994) contextual choice model. Preference for the alternative that delivered more frequent terminal links was usually stronger in the first few sessions of a condition than at the end of a condition, suggesting that the conditioned reinforcing effect of the additional terminal-link presentations was, in part, transitory.

In Experiment 1, the choice responses of 8 pigeons were observed during 50 periods of transition. Each condition began with equal probabilities of reinforcement on 2 response keys and switched to unequal probabilities. With the ratio of the 2 probabilities held constant, preference for the higher probability developed more rapidly when the 2 probabilities were high than when they were low. In Experiment 2, each condition began with 2 equal variable-interval schedules, but later 1 key delivered 60%, 75%, or 90% of the reinforcers. The rate of approach to asymptotic performance was roughly the same with all 3 reinforcement percentages. These and previous results pose difficulties for some well-known models of acquisition, but the results are well described by a simple model that states that the strength of each response is independently increased by reinforcement and decreased by nonreinforcement.

Two experiments measured pigeons' choices between probabilistic reinforcers and certain but delayed reinforcers. In Experiment 1, a peck on a red key led to a 5-s delay and then a possible reinforcer (with a probability of .2). A peck on a green key led to a certain reinforcer after an adjusting delay. This delay was adjusted over trials so as to estimate an indifference point, or a duration at which the two alternatives were chosen about equally often. In all conditions, red houselights were present during the 5-s delay on reinforced trials with the probabilistic alternative, but the houselight colors on nonreinforced trials differed across conditions. Subjects showed a stronger preference for the probabilistic alternative when the houselights were a different color (white or blue) during the delay on nonreinforced trials than when they were red on both reinforced and nonreinforced trials. These results supported the hypothesis that the value or effectiveness of a probabilistic reinforcer is inversely related to the cumulative time per reinforcer spent in the presence of stimuli associated with the probabilistic alternative. Experiment 2 tested some quantitative versions of this hypothesis by varying the delay for the probabilistic alternative (either 0 s or 2 s) and the probability of reinforcement (from .1 to 1.0). The results were best described by an equation that took into account both the cumulative durations of stimuli associated with the probabilistic reinforcer and the variability in these durations from one reinforcer to the next.

In three experiments, pigeons chose between a small amount of food delivered after a short delay and a larger amount delivered after a longer delay. A discrete-trial adjusting-delay procedure was used to estimate indifference points-pairs of delay-amount combinations that were chosen about equally often. In Experiment 1, when additional reinforcers were available during intertrial intervals on a variable-interval schedule, preference for the smaller, more immediate reinforcer increased. Experiment 2 found that this shift in preference occurred partly because the variable-interval schedule started sooner after the smaller, more immediate reinforcer, but there was still a small shift in preference when the durations and temporal locations of the variable-interval schedules were identical for both alternatives. Experiment 3 found greater increases in preference for the smaller, more immediate reinforcer with a variable-interval 15-s schedule than with a variable-interval 90-s schedule. The results were generally consistent with a model that states that the impact of any event that follows a choice response declines according to a hyperbolic function with increasing time since the moment of choice.

The hyperbolic-decay model is a mathematical expression of the relation between delay and reinforcer value. The model has been used to predict choices in discrete-trial experiments on delay-amount tradeoffs, on preference for variable over fixed delays, and on probabilistic reinforcement. Experiments manipulating the presence or absence of conditioned reinforcers on trials that end without primary reinforcement have provided evidence that the hyperbolic-decay model actually predicts the strength of conditioned reinforcers rather than the strength of delayed primary reinforcers. The model states that the strength of a conditioned reinforcer is inversely related to the time spent in its presence before a primary reinforcer is delivered. A possible way to integrate the model with Grace's (1994) contextual-choice model for concurrent-chain schedules is presented. Also discussed are unresolved difficulties in determining exactly when a stimulus will or will not serve as a conditioned reinforcer.

In an adjusting-delay choice procedure, pigeons could peck on either a red key or a green key. A peck on the red key always led to a delay associated with red houselights and then food. The delay was adjusted over trials to estimate an indifference point-a delay at which the two keys were chosen about equally often. In some conditions, a peck on the green key led to food on all trials after delays of either 10 a or 30 s, and green houselights were lit during the delays. In other conditions, food was presented on only half of the green-key trials. If the green houselights continued to occur on both reinforcement and nonreinforcement trials, preference for the green key always decreased. Preference for the green key also decreased if half of the trials had 30-s houselights followed by food and the other half had no green houselights and no food. However, preference for the green key actually increased if half of the trials had 10-s green houselights followed by food and the other half had no green houselights followed by no food. The latter condition therefore demonstrated a case in which preference for an alternative increased when food was removed from half of the trials. The results suggest that the red and green houselights served as conditioned reinforcers. A hyperbolic decay model (Mazur, 1989) provided good predictions for all conditions by assuming that the strength of a conditioned reinforcer is inversely related to the total time spent in its presence before food is delivered.

Pigeons' responses on two keys were recorded before and after the percentage of reinforcers delivered by each key was changed. In each condition of Experiment 1, the reinforcement percentage for one key was 50% for several sessions, then either 70% or 90% for one, two, or three sessions, and then 50% for another few sessions. At the start of the second and third sessions after a change in reinforcement percentages, choice percentages often exhibited spontaneous recovery-a reversion to the response percentages of earlier sessions. The spontaneous recovery consisted of a shift toward a more extreme response percentage in some cases and toward a less extreme response percentage in other cases, depending on what reinforcement percentages were previously in effect. In Experiment 2, some conditions included a 3-day rest period before a change in reinforcement percentages, and other conditions included no such rest days. Slightly less spontaneous recovery was observed in conditions with the rest periods, suggesting that the influence of prior sessions diminished with the passage of time. The results are consistent with the view that choice behavior at the start of a new session is based on a weighted average of the events of the past several sessions.

Two experiments studied the phenomenon of procrastination, in which pigeons chose a larger, more delayed response requirement over a smaller, more immediate response requirement. The response requirements were fixed-interval schedules that did not lead to an immediate food reinforcer, but that interrupted a 55-s period in which food was delivered at random times. The experiments used an adjusting-delay procedure in which the delay to the start of one fixed-interval requirement was varied over trials to estimate an indifference point-a delay at which the two alternatives were chosen about equally often. Experiment 1 found that as the delay to a shorter fixed-interval requirement was increased, the adjusting delay to a longer fixed-interval requirement also increased, and the rate of increase depended on the duration of the longer fixed-interval requirement. Experiment 2 found a strong preference for a fixed delay of 10 s to the start of a fixed-interval requirement compared to a mixed delay of either 0 or 20 s. The results help to distinguish among different equations that might describe the decreasing effectiveness of a response requirement with increasing delay, and they suggest that delayed reinforcers and delayed response requirements have symmetrical but opposite effects on choice.

A discrete-trials adjusting-delay procedure was used to investigate the conditions under which pigeons might show a preference for partial reinforcement over 100% reinforcement, an effect reported in a number of previous experiments. A peck on a red key always led to a delay with red houselights and then food. In each condition, the duration of the red-houselight delay was adjusted to estimate an indifference point. In 100% reinforcement conditions, a peck on a green key always led to a delay with green houselights and then food. In partial-reinforcement conditions, a peck on the green key led either to the green houselights and food or to white houselights and no food. In some phases of the experiment, statistically significant preference for partial reinforcement over 100% reinforcement was found, but this effect was observed in only about half of the pigeons. The effect was largely eliminated when variability in the delay stimulus colors was equated for 50% reinforcement conditions and 100% reinforcement conditions. Idiosyncratic preferences for certain colors or for stimulus variability may be at least partially responsible for the effect.

In three experiments, pigeons chose between two alternatives that differed in the probability of reinforcement and the delay to reinforcement. A peck at a red key led to a delay of 5 s and then a possible reinforcer. A peck at a green key led to an adjusting delay and then a certain reinforcer. This delay was adjusted over trials so as to estimate an indifference point, or a duration at which the two alternatives were chosen about equally often. In Experiments 1 and 2, the intertrial interval was varied across conditions, and these variations had no systematic effects on choice. In Experiment 3, the stimuli that followed a choice of the red key differed across conditions. In some conditions, a red houselight was presented for 5 s after each choice of the red key. In other conditions, the red houselight was present on reinforced trials but not on nonreinforced trials. Subjects exhibited greater preference for the red key in the latter case. The results were used to evaluate four different theories of probabilistic reinforcement. The results were most consistent with the view that the value or effectiveness of a probabilistic reinforcer is determined by the total time per reinforcer spent in the presence of stimuli associated with the probabilistic alternative. According to this view, probabilistic reinforcers are analogous to reinforcers that are delivered after variable delays. 1989 Society for the Experimental Analysis of Behavior

In a discrete-trials procedure with pigeons, a response on a green key led to a 4-s delay (during which green houselights were lit) and then a reinforcer might or might not be delivered. A response on a red key led to a delay of adjustable duration (during which red houselights were lit) and then a certain reinforcer. The delay was adjusted so as to estimate an indifference point-a duration for which the two alternatives were equally preferred. Once the green key was chosen, a subject had to continue to respond on the green key until a reinforcer was delivered. Each response on the green key, plus the 4-s delay that followed every response, was called one “link” of the green-key schedule. Subjects showed much greater preference for the green key when the number of links before reinforcement was variable (averaging four) than when it was fixed (always exactly four). These findings are consistent with the view that probabilistic reinforcers are analogous to reinforcers delivered after variable delays. When successive links were separated by 4-s or 8-s “interlink intervals” with white houselights, preference for the probabilistic alternative decreased somewhat for 2 subjects but was unaffected for the other 2 subjects. When the interlink intervals had the same green houselights that were present during the 4-s delays, preference for the green key decreased substantially for all subjects. These results provided mixed support for the view that preference for a probabilistic reinforcer is inversely related to the duration of conditioned reinforcers that precede the delivery of food.

Twenty acquisition curves were obtained from each of 8 pigeons in a free-operant choice procedure. Every condition began with a phase in which two response keys had equal probabilities of reinforcement, and, as a result, subjects' responses were divided fairly evenly between the two keys. This was followed by a phase in which one key had a higher probability of reinforcement than the other, and the development of preference was observed. In all but a few cases, response proportions increased for the key with the higher probability of reinforcement. In most conditions, the two probabilities differed by .06, but the actual probabilities varied (from .16 and .10 in one condition to .07 and .01 in another). Development of preference for the key with the higher probability of reinforcement was slower when the ratio of the two reinforcement probabilities was small (.16/.10) than when it was large (.07/.01). This finding is inconsistent with the predictions of several different quantitative models of acquisition, including the kinetic model (Myerson & Miezin, 1980) and the ratio-invariance model (Horner & Staddon, 1987). However, the finding is consistent with a hypothesis based on Weber's law, which states that the two alternatives are more discriminable when the ratio of their reinforcement probabilities is larger, and, as a result, the acquisition of preference is faster.

This experiment measured pigeons' choices between delayed reinforcers and fixed‐ratio schedules in which a force of approximately 0.48 N was needed to operate the response key. In ratio‐delay conditions, subjects chose between a fixed‐ratio schedule and an adjusting delay. The delay was increased or decreased several times a session in order to estimate an indifference point—a delay duration at which the two alternatives were chosen about equally often. Each ratio‐delay condition was followed by a delay‐delay condition in which subjects chose between the adjusting delay and a variable‐time schedule, with the components of this schedule selected to match the ratio completion times of the preceding ratio‐delay condition. The adjusting delays at the indifference point were longer when the alternative was a fixed‐ratio schedule than when it was a matched variable‐time schedule, which indicated a preference for the matched variable‐time schedules over the fixed‐ratio schedules. This preference increased in a nonlinear manner with increasing ratio size. This nonlinearity was inconsistent with a theory that states that indifference points for both time and ratio schedules can be predicted by multiplying the choice response‐reinforcer intervals of the two types of schedules by different multiplicative constants. Two other theories, which predict nonlinear increases in preference for the matched variable‐time schedules, are discussed. 1990 Society for the Experimental Analysis of Behavior

Ten acquisition curves were obtained from each of 4 pigeons in a two‐choice discrete‐trial procedure. In each of these 10 conditions, the two response keys initially had equal probabilities of reinforcement, and subjects' choice responses were about equally divided between the two keys. Then the reinforcement probabilities were changed so that one key had a higher probability of reinforcement (the left key in half of the conditions and the right key in the other half), and in nearly every case the subjects developed a preference for this key. The rate of acquisition of preference for this key was faster when the ratio of the two reinforcement probabilities was higher. For instance, acquisition of preference was faster in conditions with reinforcement probabilities of .12 and .02 than in conditions with reinforcement probabilities of .40 and .30, even though the pairs of probabilities differed by .10 in both cases. These results were used to evaluate the predictions of some theories of transitional behavior in choice situations. A trial‐by‐trial analysis of individual responses and reinforcers suggested that reinforcement had both short‐term and long‐term effects on choice. The short‐term effect was an increased probability of returning to the same key on the one or two trials following a reinforcer. The long‐term effect was a gradual increase in the proportion of responses on the key with the higher probability of reinforcement, an increase that usually continued for several hundred trials. 1990 Society for the Experimental Analysis of Behavior