Review

DOI: http://dx.doi.org/10.21500/20112084.2284

An Animal Model of Human Gambling

Un modelo animal de juegos de azar humanos

Thomas Zentall^a*

^a Faculty of Psychology, University of Kentucky, Lexington, KY, United States of America.

Article history:

Received: 18-03-2016. Revised: 23-05-2016. Accepted: 26-05-2016

* Corresponding author: Thomas R. Zentall, Department of Psychology, University of Kentucky, Lexington, KY 40506, United States of America. Email address: zentall@uky.edu.

ABSTRACT

Human gambling generally involves taking a risk on a low probability high outcome alternative over the more economically optimal high probability low outcome alternative (not gambling). Surprisingly, although optimal foraging theory suggests that animals should be sensitive to the overall probability of reinforcement, the results of many experiments suggest otherwise. For example, they do not prefer an alternative that 100% of the time provides them with a stimulus that always predicts reinforcement over an alternative that provides them with a stimulus that predicts reinforcement 50% of the time. This line of research leads to the conclusion that preference depends on the predictive value of the stimulus that follows and surprisingly, not on its frequency. A similar mechanism likely accounts for the suboptimal choice that humans have to engage in commercial gambling.

Key words: gambling, animal model, reinforcement and risk taking.

RESUMEN

En los humanos, los juegos de azar generalmente implican la toma de riesgo en alternativas de baja probabilidad y buenos resultados, sobre alternativas más probables y de resultados bajos pero económicamente óptimos (no apostar). Lo sorprendente es, que aunque la teoría de forrajeo optimo sugiere que los animales deberían ser sensibles a una probabilidad general de reforzamiento, los resultados de muchos experimentos sugieren lo contrario. Por ejemplo, los animales no prefieren una alternativa que en un 100% de las veces provea un estímulo que siempre prediga el reforzamiento por encima de una alternativa que provea un estímulo que prediga el reforzamiento en un 50% de las veces. Esta línea de investigación lleva a la conclusión de que las preferencias dependen del valor predictivo de los estímulos que los preceden, y no de su frecuencia como era de esperarse. Un mecanismo similar probablemente explicaría la elección sub-óptima que los humanos realizan en los juegos de azar comerciales.

Palabras clave: juegos de azar, modelo animal, reforzamiento y toma de riesgo.

1. THE BEHAVIORAL BASIS OF HUMAN GAMBLING

1.1 Introduction

Commercial gambling can be defined as those forms of gambling in which the average net return is less than what is wagered. Such gambles are typical of casino games such as slot machines, roulette, and black-jack, and are especially true of lotteries. Commercial gambling is characterized by a low probability of a high payoff outcome in which the choice to abstain from gambling is associated with the more optimal, high probability low payoff outcome. Thus, when humans engage in commercial gambling they are making suboptimal choices, however, in spite of the poor odds of winning over losing, it is possible for one to justify the behavior in terms of the excitement or pleasure derived from the activity. Those who gamble often describe the activity as a form of entertainment, yet for those who gamble excessively, it can have serious effects on one's career and family relations, not to mention one's financial security. In fact, problem gambling is recognized clinically as an impulse control disorder in which people show impaired behavioral inhibition and a failure to consider the long-term consequences of the decisions they make ( DSM-5, 2013).

In addition to what gamblers describe as the pleasure of gambling, most public gambling (e.g., casinos) when someone wins, it is more salient than when someone loses (bells ring and lights flash at casinos when someone wins big and big winners of lotteries are often mentioned on the news). This is sometimes referred to as an example of the availability heuristic ( Tversky & Kahneman, 1974).

Interestingly, examination of the behavioral ecology literature suggests that one should not find evidence of suboptimal choice in nonhuman animals because natural selection should have selected animals to be optimal foragers (Pyke, Pulliam, & Charnow, 1977; Stephens & Krebs, 1986). Given appropriate experience, nonhuman animals are presumed to be sensitive to the relative amounts of food obtained from different alternatives or patches (see Fantino & Abarca, 1985). For these reasons, one would not expect other animals to be as susceptible to making suboptimal choices as humans.

2. A PIGEON MODEL OF HUMAN GAMBLING BEHAVIOR

Pigeons have been found to prefer choices that produce discriminative stimuli over those that do not. Specifically, they prefer choices that sometimes result in a strong conditioned reinforcer (followed by reinforcement 100% of the time) and some-times result in a strong conditioned inhibitor (never followed by reinforcement) over those that result in weak conditioned reinforcers (both followed by reinforcement 50% of the time) even though choice of either alternative would result in the same amount of reinforcement (see Fig. 1; Roper & Zentall, 1999). In the same study it was found that the pigeons were willing to work considerably harder for the discriminative stimuli than for the nondiscriminative stimuli. When working harder involves added pecking, it implies a longer delay to reinforcement and research on delay discounting implies that waiting longer serves to devalue the magnitude of reinforcement (Rachlin & Green, 1972). A more direct demonstration of the devaluation of reinforcement associated with nondiscriminative stimuli was demonstrated by several investigators ( Belke & Spetch, 1994 ; Fantino, Dunn, & Meck., 1979; Kendall, 1974 , 1985; Mazur, 1996; Spetch, Belke, Barnet, Dunn, & Pierce, 1990; Spetch, Mondloch, Belke, & Dunn, 1994). In that research, pigeons were given a choice between an alternative that provided discriminative stimuli, one associated with 100% reinforcement and the other with the absence of reinforcement (each occurring 50% of the time), and an alternative that always provided a stimulus associated with 100% reinforcement (see Fig. 2). Although the results were not always consistent, the pigeons did not generally prefer the alternative that provided them with 100% reinforcement and many of the pigeons actually preferred the alternative that provided 50% reinforcement. In a more recent experiment, we produced more consistent preferences for the suboptimal alternative by decreasing the probability of reinforcement associated with the optimal alternative from 100% to 75% reinforcement (see Fig 3; Gipson, Alessandri, Miller, & Zentall, 2009). And in a follow-up study ( Stagner & Zentall, 2010) we found that if we reduced the probability of reinforcement associated with the discriminative stimulus alternative, we could obtain an even larger preference for that alternative ( Stagner & Zentall, 2010). Specifically, the probability of reinforcement associated with the discriminative stimulus alternative was only 0.20 (the stimulus that reliably predicted reinforcement occurred on only 20% of the trials), whereas the probability of reinforcement associated with the nondiscriminative stimulus alternative was 0.50 (2.5 times the probability reinforcement associated with the discriminative stimulus alternative, see Fig. 4). Under these conditions the pigeons showed a very strong preference for the discriminative stimulus alternative.

3. THE VALUE OF THE SUBOPTIMAL CHOICE

Given that delay discounting can be used as a means of reducing the value of an alternative, in all of the research described so far, the time from choice of the alternative to the outcome (reinforcement or its absence) was held constant. However, one way to assess the value of the discriminative stimulus alternative is to ask how much shorter the delay to reinforcement must be following choice of the alternative associated with the nondiscriminative stimuli for pigeons to shift their preference to that alternative. In this experiment we used a procedure similar to that described by Mazur (1996) in which the delay to reinforcement for the two alternatives starts out the same but is gradually reduced for the nondiscriminative alternative until the preference switches and then the delay to reinforcement for the nondiscriminative alternative is gradually increased until it switches back to the discriminative stimulus alternative (Zentall & Stagner, 2011b, Exp. 1). In this experiment the probability of reinforcement associated with the suboptimal alternative was 20% and the optimal alternative was 50%. The results of this experiment indicated that the duration of conditioned reinforcer associated with the optimal alternative had to be reduced from 10.0 s to about 4.4 s before the pigeons began to prefer it over the suboptimal alternative for which the conditioned reinforcer remained at 10.0 s. Thus, it was not until the delay to reinforcement associated with the optimal alternative was less than half of the delay to reinforcement associated with the suboptimal alternative that the pigeons no longer preferred the suboptimal alternative.

4. THE ALLAIS PARADOX

Humans often show a paradoxical choice behavior sometimes referred to as the Allais paradox ( Allais, 1953) or the certainty effect ( Shafir et al., 2008). For example, if humans are given a choice between a 100% chance of earning $5 or an 80% chance of earning $10, although the average return on the 80% chance of earning $10 is higher ($8), most people choose the certain $5. That this choice depends on the certainty of the suboptimal choice can be seen by reducing both of the probabilities by one half (i.e., a choice between a 50% chance of earning $5 and a 40% chance of earning$10). In this case, the opposite preference will typically be found. According to expected utility theory, the results of the second choice should be the same as the first choice but they are not. The reason subjects often give for the preference for the certain $5 is they would be especially disappointed if they chose the 80% chance of $10 and lost, whereas in the case of the preference for the 40% chance of obtaining $10 they reason that they could almost as easily have lost had they chosen the 50% chance of obtaining $5. If humans choose suboptimally to avoid the possibility of a loss, could that also be why pigeons choose the alternative that provides the conditioned reinforcer that predicts 100% reinforcement over the alternative that provides a conditioned reinforcer that predicts 50% reinforcement? To test this hypothesis we conducted an experiment similar to that of Stagner and Zentall (2010) in which the probabilities of reinforcement associated with the conditioned reinforcers were reduced by 20%. As in the Stagner and Zentall study, the probability of reinforcement associated with the discriminative stimulus alternative was only 0.20, however, on those trials, reinforcement occurred only 80% of the time. Thus, reinforcement was no longer certain. To maintain the same ratio of reinforcement for the optimal alternative, the probability of reinforcement associated with the nondiscriminative stimulus alternative was reduced to 0.40. Thus, the probability of reinforcement associated with the discriminative stimulus alternative was now .16 whereas the probability of reinforcement associated with the nondiscriminative stimulus alternative was now .40 (again, a ratio of 1:2.5). But once again, the pigeons showed a strong preference for the discriminative stimulus alternative ( Zentall & Stagner, 2011b). Thus, the uncertainty associated with the conditioned reinforcer that followed choice of the suboptimal alternative did not deter the pigeons from choosing suboptimally. Of course, it is possible that if the probability of reinforcement associated with low probability high payoff stimulus was reduced still further, the pigeons' choice may have reversed to become optimal. However, as we will see in the next section, certainty does not appear to be the mechanism responsible for suboptimal choice when probability of reinforcement is manipulated ( Zentall & Stagner, 2011a).

5. A BETTER ANIMAL ANALOG OF HUMAN GAMBLING BEHAVIOR

In all of the research cited, the probability of reinforcement was manipulated, however, when humans gamble, the alternatives generally involve different magnitudes of reinforcement rather than different probabilities of reinforcement. For example, a $1 lottery ticket might yield a much larger amount of money. It is possible that the effect we have been observing with the manipulation of probability of reinforcement occurs because the pigeons are avoiding an alternative that results in stimuli associated with an uncertain outcome (0.75 probability of reinforcement in Gipson et al., 2009, and 0.50 probability of reinforcement in Stagner and Zentall, 2010). If the effect that we have been studying with pigeons is a good analog of human gambling behavior, it should be possible to find a similar effect by manipulating the magnitude of reinforcement, rather than the probability of reinforcement, and removing the uncertainty of the outcome associated with the nondiscriminative stimuli.

To test this hypothesis, Zentall and Stagner (2011a) gave pigeons a choice between two alternatives. Choice of one alternative on 20% of the trials produced a stimulus that always predicted the delivery of 10 pellets of food and on the remaining 80% of the trials, produced a stimulus that always predicted the delivery of 0 pellets. Thus, this alternative was associated with an average of 2 pellets per trial (see design in Fig. 5).Choice of the other alternative always produced one of two stimuli each of which always predicted the delivery of 3 pellets. Thus, the second alternative was associated with a consistent 3 pellets per trial. Once again, if pigeons are sensitive to the amount of food they obtain over time, they should select the 3-pellet option. Contrary to this prediction, however, the pigeons showed a strong preference for the variable 2-pellet alternative over the fixed 3-pellet alternative.

But perhaps pigeons were just showing a preference for the variable option (10 pellets 20% of the time) over the fixed 3-pellet option. After all, there is evidence that that variable schedules are often preferred over fixed schedules (Fantino, 1967; Hursh & Fantino, 1974). To test the hypothesis that the outcome variability may have been responsible for the results of the Zentall and Stagner (2011a) experiment, we repeated the experiment and made the discriminative stimuli nondiscriminative. That is, choice of the alternative that provided an average of 2 pellets per trial now produced one of two stimuli, each of which was associated with a 20% chance of providing 10 pellets. The alternative that provided a consistent 3 pellets per trial continued to do so. Under these conditions, the pigeons quickly learned to choose optimally. That is, they now showed a strong preference for the alternative associated with 3 pellets per trial. Thus, it was not the variability of reinforcement associated with the 20% reinforcement alternative that was responsible for the preference for that alternative but the discriminative stimuli that followed that choice.

6. WHAT IS RESPONSIBLE FOR THE SUBOPTIMAL CHOICE BY PIGEONS?

6.1 Insufficient conditioned inhibition

Although it is clear that the stimulus associated with reinforcement that follows choice of the suboptimal alternative becomes a strong conditioned reinforcer, it is not clear why the stimulus associated with the absence of reinforcement (the S-) that follows choice of the same suboptimal alternative does not become a sufficiently strong conditioned inhibitor (especially given that it occurs four times as often) to reduce the preference for that alternative.

One possibility is that once the S- stimulus is identified, the pigeon turns away from it, thus limiting its effectiveness (i.e., it maintained little observing behavior; see Dinsmoor, 1985). For example, Roberts (1972) has shown that the effectiveness of a discriminative stimulus is directly related to the duration that it is observed. Consistent with this possibility, the pigeons in Gipson et al. (2009), Stagner and Zentall (2010), and Zentall and Stagner (2011a, b) rarely pecked at the S−, whereas in each of those experiments they pecked at all of the stimuli that were followed by reinforcement. Interestingly, however, Dinsmoor found that when pigeons were presented with an S− and they were able to turn it off (but turning it off did not change the schedule of reinforcement that was in effect), they did so. Thus, the S− stimulus did appear to have some inhibitory properties.

To test the hypothesis that pigeons may have reduced the inhibitory effects of the S- stimulus by turning away from it, Stagner, Laude, and Zentall (2011) used a diffuse houselight as the S− stimulus. If the failure to observe or remain in the presence of the S− stimulus is responsible for the preference for the alternative providing less reinforcement, pigeons exposed to a diffuse stimulus that signals the absence of reinforcement should develop more inhibition to the S− and thus, should show a greater preference for the alternative associated with the optimal alternative. When we conducted this study, we found that the pigeons continued to prefer the discriminative stimulus alternative associated with an overall lower probability of reinforcement, and did so similar to controls for which the diffuse stimulus served as the S+ (the conditioned reinforcer) as well as for controls for which neither the S+ nor the S− was a diffuse stimulus.

Hearst Besley, and Farthing (1970) described more direct procedures to assess the inhibition associated with a stimulus that is followed by the absence of reinforcement. One of the procedures involves the presentation of a compound consisting of a known conditioned reinforcer (S+), together with the presumed conditioned inhibitor. Evidence for conditioned inhibition is found when responding to the S+ decreases when the S− is presented in compound with it. To devise such a test with the Stagner and Zentall (2010) design (20% vs. 50% reinforcement) we used a shape (a vertical line) as the S− rather than a color ( Laude, Stagner, & Zentall, 2014). We then tested the pigeons with both the S+ and the S+/S− compound, first early in training before the appearance of suboptimal choice and again later in training after the appearance of suboptimal choice. To assess the generality of the effect, we used both a repeated measures design and a between groups design. Consistent with the hypothesis that the S− loses its inhibitory strength with training, early in training we found that combining the S− with the S+ resulted in a large drop in responding relative to the S+ alone, whereas later in training there was a significantly smaller drop in responding.

Consistent with these results, a theory based on the absence of conditioned inhibition to losses also has been proposed to account for human gambling behavior. For example, Breen and Zuckerman (1999) reported that humans who gamble regularly have been found to attend more to their wins than occasional gamblers and less to their considerably more frequent losses. Similarly, research with humans has found that problem gamblers show reduced sensitivity to aversive conditioning ( Brunborg et al., 2010) which should also serve to inhibit behavior.

6.2 Reduced response cost associated with the S- stimulus.

A different explanation for why pigeons prefer 20% signaled over 50% unsignaled reinforcement has to do with the response that the pigeons make to the stimuli that follow choice. In the case of the suboptimal choice, the pigeons peck at the S+ stimulus but peck little or not at all at the S- stimulus. Thus, almost all pecking is reinforced. In the case of the optimal choice, however, pecking occurs on all trials to the stimulus that predicts reinforcement 50% of the time but that pecking is reinforced on only half of the trials. Thus, there is considerable nonreinforced pecking following choice of the optimal alternative but very little nonreinforced pecking following choice of the suboptimal alternative. This hypothesis suggests that the preference for the suboptimal alternative occurs because the S+ that follows choice of the optimal alternative has less value than the S+ that follows choice of the suboptimal alternative and because there is little pecking to the S- that follows choice of the suboptimal alternative, it contributes little inhibition to that choice. Although this hypothesis provides a reasonable account of the data from Gipson et al. (2009) and Stagner and Zentall (2010) it cannot account for the results of the experiments in which the optimal alternative is followed by reinforcement 100% of the time ( Belke & Spetch, 1994 ; Fantino et al., 1979; Mazur, 1996; Spetch et al., 1990, 1994; Zentall & Stagner, 2011a).

7. CHOICE IS DETERMINED BY THE VALUE OF THE S+ STIMULUS

If the inhibitory value of the conditioned inhibitory stimulus plays a minimal role in choice of the suboptimal alternative, then it must be the value of the conditioned reinforcer, rather than the overall probability of reinforcement associated with choice of each alternative, that is responsible for suboptimal choice. Thus, in the Gipson et al. (2009) study, pigeons do not appear to be choosing between 50% reinforcement associated with the suboptimal alternative and 75% reinforcement associated with the optimal alternative, rather it appears that they are choosing between the conditioned reinforcer associated with 100% reinforcement and the conditioned reinforcer associated with 75% reinforcement. Similarly, in the Stagner and Zentall (2010) study, pigeons do not appear to be choosing between 20% reinforcement associated with the suboptimal alternative and 50% reinforcement associated with the optimal alternative. Instead they appear to be choosing between the conditioned reinforcer associated with 100% reinforcement and the conditioned reinforcer associated with 50% reinforcement. Likewise, in the Zentall and Stagner (2011a , b) study pigeons appear to choose between the conditioned reinforcer associated with 10 pellets and the conditioned reinforcer associated with 3 pellets. This analysis assumes that it is the value of the conditioned reinforcer that follows choice, rather than the frequency of reinforcement associated with that choice, that determines whether the pigeons will choose suboptimally or not and it leads to in interesting prediction. If pigeons are given a choice between two alternatives, one which leads to discriminative stimuli with the S+ occurring 50% of the time (and the S− occurring 50% of the time) and the other which leads to an S+ that occurs 100% of the time, the pigeons should be relatively indifferent between the two alternatives. Interestingly, there is published support for this prediction (Belke & Spetch, 1994; Fantino et al., 1979; Mazur, 1996; Spetch et al., 1990, 1994). Although generally it has been found that some pigeons prefer the alternative that provides 50% reinforcement, others prefer the optimal alternative that that provides 100% reinforcement. For example, Spetch et al. (1990, Exp. 1) found that two of their four pigeons chose the suboptimal alternative more than the optimal alternative and Belke and Spetch found that two of their eight pigeons actually preferred the optimal alternative. However, what appear to be individual differences in optimal versus suboptimal choice actually may be influenced by an artifact present in the initial choice alternatives. In all of these experiments, the fact that the choice was between two alternatives that could be discriminated solely by their spatial location may result in a confound of schedule preference and idiosyncratic spatial preference.

In a study by Stagner et al. (2012), we tested this hypothesis and the hypothesis that independent of the probability of their occurrence, conditioned reinforcers with comparable value would result in indifference between the two alternatives. In that study, pigeons were given a choice between two alternatives, one which led to discriminative stimuli with the S+ occurring 20% of the time (and the S− occurring 80% of the time) and the other which led to discriminative stimuli with the S+ occurring 50% of the time (and the S− occurring 50% of the time). That is, in this study, both alternatives were associated with conditioned reinforcers predicting reinforcement 100% of the time but in one case the conditioned reinforcer occurred two and a half times as often as the other. To avoid spurious spatial preferences that might be attributed to schedule preferences, to signal the two initial alternatives we used line orientation stimuli (vertical or horizontal lines; see Fig. 6), the location of which (left or right) varied randomly from trial to trial. The results confirmed our prediction. All of the pigeons were virtually indifferent between the two alternatives. To confirm that the pigeons were capable of discriminating between the two schedules, when the stimuli that followed the alternative associated with 50% reinforcement were made nondiscriminative, a strong suboptimal choice effect was found, whereas when the stimuli that followed both alternatives were made nondiscriminative, a strong optimal choice effect was found. Thus, the value of conditioned reinforcers play an important role in the suboptimal choice found for pigeons but the frequencies of those stimuli appear to have much less of an effect on schedule choice. Recently we found convergent support for the stimulus value hypothesis using a design more similar to that used by Belke and Spetch (1994), Fantino et al. (1979), Mazur (1996), Spetch et al. (1990 , 1994), and others in which pigeons were given a choice between 50% signaled reinforcement and 100% signaled reinforcement. In this experiment ( Smith & Zentall, in press) the main difference from the earlier research was the two alternatives were differentiated by distinctive shapes that varied in their location from trial to trial (see Fig. 7). Once again, in spite of the fact that the optimal alternative was associated with twice as much reinforcement as the suboptimal alternative, we found indifference between the two alternatives. As a further test of the stimulus value hypothesis we recently conducted an experiment in which only two terminal link stimuli were used (Smith, Bailey, Chow, Beckmann, & Zentall, in press). In Phase 1 of the first experiment, the initial link alternatives were each associated with 50% reinforcement but the predictive alternative was followed by a terminal link stimulus 50% of the time that was followed by reinforcement 100% of the time, whereas the nonpredictive alternative was always followed by a terminal link stimulus that was followed by reinforcement only 50% of the time (Fig. 8). Not surprisingly and consistent with the results of Roper and Zentall (1999), the pigeons showed a strong preference for the predictive alternative. In Phase 2, however, we reduced the frequency of the discriminative stimulus from 50% to 25% and in Phase 3 to 12.5% and found no reduction in preference for the discriminative alternative. Thus, the pigeons preferred the discriminative alternative even though the nondiscriminative alternative provided four times as much food (see also Vasconcelos, Monteiro, & Kacelnik, 2015). In Experiment 2, we reversed the initial link stimuli that signaled the predictive and nonpredictive alternatives and returned the probability of reinforcement associated with each to 50%. Once we had recovered the pigeons' preference for the predictive alternative we increased the probability of reinforcement for choice of the nonpredictive alternative to 75% in Phase 2 and then to 100% in Phase 3 (at this point the nonpredictive alternative was now predictive as well). Although the preference for the predictive alternative did not decline when the probability of reinforcement for choice of the nonpredictive alternative increased to 75%, it did decline when the probability of reinforcement for choice of the nonpredictive alternative increased to 100% but it remained significantly above chance.

According to the stimulus value hypothesis, when the probability of reinforcement for choice of the nonpredictive alternative increased to 100% the pigeons should have been indifferent between the two alternatives because now the two terminal link stimuli had equal value, however, it is likely that carryover effects from Phases 1 and 2 of training could account for the absence of indifference. In any case, the frequency of the terminal link stimuli played little role in the pigeons' preferences.

8. THE ROLE OF THE TERMINAL LINK DURATION

In all of our research (with the exception of Gipson et al, 2009) we have used a terminal link duration of 10 s. However, with any procedure in which choice of the optimal alternative is associated with less than 100% reinforcement, there is reason to believe that the longer the terminal link duration, the greater should be the preference for the suboptimal alternative. The reasoning is as follows: although lengthening the terminal link stimulus should make both alternatives less appealing because of delayed reinforcement, the immediate appearance of the predictive stimulus signaling reinforcement on some trials following choice of the suboptimal alternative should provide conditioned reinforcement, whereas the uncertainty of reinforcement following choice of the optimal alternative should have little conditioned reinforcing effect. Interestingly, even when there is no uncertainty associated with the outcome of the trial following choice of the optimal alternative (100% reinforcement) there is evidence that increasing the duration of the terminal link stimuli from 5 to 90 s results in an increase in preference for the suboptimal alternative from an average 17% at the shortest terminal link duration to 67% at the longest ( Spetch et al., 1990; Spetch et al., 1994). Thus, terminal link duration, in addition to the value of the terminal link stimuli may affect preference for the suboptimal alternative.

For pigeons, the value of conditioned reinforcer appears to strongly influence choice of the alternative that preceded it but conditioned reinforcers also appear to play an important role in human suboptimal choice. This conclusion is supported by a line of research on observing behavior which shows that humans will work to obtain a signal for reinforcement but not a signal for the absence of reinforcement when neither changes the probability of reinforcement. For example, Fantino and Case (1983; see also Fantino & Silberberg, 2010) exposed subjects to a mixed, variable time (response independent), extinction schedule in which in one condition, responses produced a stimulus which signaled that the variable time schedule was in effect (a presumed conditioned reinforcer) or at other times, a stimulus which signaled that the extinction schedule was in effect (a presumed conditioned inhibitor). In a second condition, responses produced only the presumed conditioned reinforcer (when it would have been in effect) and in a third condition, responses produced only the presumed conditioned inhibitor (when it would have been in effect). Importantly, in no case did responding have any effect of the schedule itself; it only identified the schedule that was already in effect. Fantino and Case found that subjects would respond to produce the stimulus associated with reinforcement but would not respond to produce the signal for nonreinforcement, even though they both contain the same amount of information (i.e., a conditioned reinforcer let the subject know that reinforcement could be obtained, whereas a conditioned inhibitor let the subject know that reinforcement could not be obtained). The results of these experiments with pigeons and humans are consistent with the findings from human gambling research that conditioned reinforcers play an important role for problem gamblers (Crockford et al., 2005), whereas conditioned inhibitors exert very little control over the their decisions to gamble (Field and Cox, 2008 ; Franken et al., 2003; Holst et al., 2010; Tversky and Kahneman, 1974). The results of these experiments suggest that one approach to the treatment of pathological gamblers might be to make them more aware of their losses by making their losses more salient.

9. THE IMMEDIACY OF THE TERMINAL LINK STIMULI

According to the stimulus value hypothesis, indifference between the optimal and suboptimal alternatives results from the similar value of the terminal link stimuli that predict reinforcement (both 100% of the time). However, in spite of the indifference between initial link alternatives found by Smith & Zentall (2016) and Stagner et al. (2012) there is evidence that there are important differences between those two predicative stimuli. Using the procedure depicted in Figure 2, McDevitt, Spetch, and Dunn (1997) inserted a dark 5-s gap selectively prior to the onset of each terminal link stimulus. When the gap was inserted following choice of the suboptimal alternative prior to the onset of the S- stimulus, it had little effect on the preference for the suboptimal alternative, whereas when the gap was inserted prior to the onset of the S+ stimulus, it resulted in a large reduction in the preference for the suboptimal alternative. Surprisingly, however, when the gap was inserted prior to the onset of the S+ stimulus that followed choice of the optimal alternative it had little effect on the preference for the suboptimal alternative (see also Vasconcelos et al., 2015). According to the stimulus value hypothesis, as both S+ stimuli should have had similar value, the gap should have similarly reduced the preference for the alternative that preceded it. McDevitt et al. (2016) reasoned that uncertainty enhances the value of the stimulus that resolves it. This account is similar to that suggested by Gipson et al., (2009) and Zentall and Stagner (2011a) who proposed that preference for the suboptimal alternative resulted from positive contrast between the relatively low value of the suboptimal alternative (50% reinforcement) prior to the appearance of the terminal link stimulus, and the appearance of the conditioned reinforcer, when it occurred. Alternatively, choice of the optimal alternative would not be expected to result in contrast upon presentation of the terminal link stimulus. Based on Fantino's delay reduction theory (DRT, Fantino, 1969 ; Fantino, Preston & Dunn, 1993), Dunn and Spetch (1990) proposed that any event that signals a reduction in the delay beyond that already signaled by the choice response will function as a conditioned reinforcer. Their hypothesis also assumes that events that signal an increase in the delay beyond that already signaled by the choice response (i.e., the signal for the absence of reinforcement) will play no role in choice of the alternative that preceded it. McDevitt et al. (2016) refer to this explanation as the Signals for Good News (SiGN) hypothesis.According to both the contrast account and the SiGN hypothesis, it is the relative change in value of the signal for reinforcement that is responsible for the preference for the suboptimal alternative. Although both hypotheses explain why the signal for reinforcement that follows the suboptimal alternative would be preferred over the signal for reinforcement that follows the optimal alternative, it is not clear how that difference can overcome the inherent bias in primary reinforcement that should be associated with the optimal alternative. That is, the contrast or good news associated with the terminal link stimulus assumes that the suboptimal initial link alternative must be of lesser value than the optimal alternative. But if it is of lesser value, it should not be chosen over the optimal alternative. Perhaps it acquires greater value through its association with the terminal link conditioned reinforcer but if the suboptimal alternative acquires greater value, the increased value should reduce the contrast or good news that follows. The SiGN hypothesis overcomes this circularity by positing that primary reinforcement associated with the initial link alternatives and conditioned reinforcement associated with the good news are independent and additive. Thus, when the good news or conditioned reinforcement exceeds the differential primary reinforcement associated with two initial link alternatives, the suboptimal alternative should be chosen; however, when they are of equal value (Smith & Zentall, in press; Stagner et al., 2012) indifference between the two alternatives should be found.

10. THE RELATION BETWEEN HUMAN GAMBLING AND THE PIGEON SUBOPTIMAL CHOICE TASK

Although there are differences between the procedures involved in human gambling decisions and the procedures used with pigeons, we propose that the underlying processes may be quite similar. That this task is analogous to human gambling would be better supported if one could show that the performance by humans on a similar task was correlated with the degree to which those individuals engaged in gambling behavior. Using a modified version of the pigeon task used by Zentall and Stagner (2011a), Molet et al. (2012) tested this prediction and found that humans who self-reported that they regularly engaged in commercial gambling, chose the suboptimal alternative significantly more than nongamblers. The results of this experiment suggest that mechanisms found to be involved in suboptimal choice by pigeons may also be relevant to human gambling.

10.1 Task Differences

When humans gamble, one can think of the choice as go/no-go because humans can choose to gamble with money that they have or refrain from gambling. Pigeons, on the other hand, choose between an optimal outcome and a suboptimal outcome, both of which involve obtaining resources that they do not already have. Although different, this distinction should make it even more likely that humans would not gamble because for humans, not only do they have a choice between a sure outcome and a probabilistic outcome but the sure outcome is immediate (money already in their pocket) whereas the probabilistic outcome is delayed by the time it takes to gamble and learn about the outcome. This may explain why only a small percentage of humans are actually problem gamblers. In fact, we have found that for pigeons, if the suboptimal outcome is delayed, relative to the optimal outcome, the pigeons begin to choose optimally (Zentall and Stagner, 2011b, Exp. 1). Because humans choose to gamble with money that they already have, unlike pigeons their losses represent actual losses rather than the absence of reinforcement. This distinction may be important because according to prospect theory (Kahneman & Tversky,1979), although gains that are certain (e.g., the certainty of winning $5) are preferred over proportionally larger gains that are probabilistic (e.g., an 80% chance of winning $10) (the certainty effect), losses that are certain (e.g., the certainty of losing $5) are avoided over proportionally larger losses that are probabilistic (e.g., an 80% chance of losing $10) (the reflection effect). That is, there is a stronger bias to win back losses than to obtain gains, an effect that typically encourages gamblers to keep gambling. Although it would be difficult to create a task in which pigeons, like humans, can choose to gamble with a reinforcer that they already have, as already noted, although gains that are certain we have tested humans who are self-reported gamblers on a version of the pigeon two-alternative choice task and found that they are more likely to choose suboptimally than self-reported non gamblers ( Molet et al., 2012). Thus, the difference between the go/no-go choice provided by commercial gambling and the two-alternative choice provided by our analog task does not appear to be responsible for the suboptimal choice by pigeons.

10.2 The role of conditioned reinforces in human gambling.

The suboptimal choice task used with pigeons involves the appearance of conditioned reinforcers following choice but prior to the appearance of the outcome. To what extent are there analogous conditioned reinforcers present in human gambling? It may not be obvious that conditioned reinforcers are present in human gambling but the results of a thought experiment suggest that they are. For example, the three reels on a slot machine can be thought of as conditioned reinforcers. Evidence that they serve that role is suggested by the likelihood that people would not likely gamble nearly as much if the reels on the slot machine could not be seen. That is, if the only outcome of money inserted in the machine would be either nothing or money falling into the coin tray. Once again, under these conditions people would be less likely to play. A similar argument can be made for other games of chance (e.g., roulette and blackjack). Thus, although there may be some procedural differences between the pigeon suboptimal choice task and human commercial gambling, the important elements of the two are quite similar as supported by the finding that a modified version of the pigeon task is able to distinguish human gambler from nongamblers ( Molet et al., 2012).

11. THE NEAR HIT EFFECT

One way in which pigeons appear to differ from humans in their preference for the suboptimal alternative is in the effect of outcomes that indicate a loss but appear to come close to winning, a near hit (sometimes referred to as a near miss). An example of a near hit outcome can best be described using a three-reel slot machine. A winning outcome consist of lining up three of the same symbols, one on each reel (e.g., three cherries). Any mixture of different symbols represents a loss but some losses are not viewed as bad by gamblers as others. For example, two cherries and a bell represent a loss but that combination appears to many gamblers to be closer to a winning outcome than a clear loss of three symbols that do not match (Reid, 1986). When MacLin, Dixon, Daugherty, and Small (2007) gave subjects a choice among three machines, one that gave near hit trials 45% of the time, another that gave near hits 30% of the time, and the third that gave near hits 15% of the time, they preferred the machine that gave near hit trials most often. Griffiths (1999) proposed that near hits encourage further game play because even though gamblers are still losing, “they must be doing something right.” Langer (1975) proposed that the near hit gave gamblers the illusion of control. That is, getting close to winning suggests that there may be skill involved in this game of chance. In games involving skill, such as shooting basket balls, near hits can provide feedback on how to modify behavior on the next trial but not in games of chance. Commercial gambling systems sometimes take advantage of the perceived value in coming close to winning by programming machines to provide more of these near hits to encourage further play.

Although there is some suggestion that rats may also show a preference for near hit trials (Winstanley, Cocker, & Rogers, 2011), the effect appeared to be one of stimulus generalization. That is, if three successive lights signaled a win (111), the rats responded equally to two lights irrespective of their order (i.e., they responded equally to 110, 101, and 011). For humans 110 would be considered a near hit, whereas 101 and 011 would be considered clear loss.

Recently, Stagner, Case, Sticklen, Duncan, and Zentall (2015) asked if pigeons preferred near hit trials over clear loss trials when the probability of reinforcement was equated (see Fig. 9). Not only did they find that pigeons preferred clear loss trials over near hit trials, but they also found that the later in the terminal link that the near hit occurred, the less they preferred the alternative with the near hit trials (Stagner et al., Exp. 2). In a follow-up experiment (Stagner et al., Exp. 3), to model the initial uncertainty of the outcome on loss trials with the three reel slot machine, choice of the near hit alternative was followed by a neutral stimulus that revealed the S+ or S- stimulus during the last 2 s of the terminal link. Once again, with the probability of reinforcement equated for the two alternatives, the pigeons preferred the alternative that provided them with clear losses over the alternative that provided them with near hits.

It is interesting to speculate as to why humans are different in this respect from pigeons. Several authors have suggested that the attraction to the near hit by humans comes from the illusion that a near hit is closer to a hit than is a loss (Griffiths, 1999 ; Reid, 1986) and Langer (1975) proposed that the near hit gives gamblers the illusion of control. After all, in many games of skill, getting closer to success does represent improvement in the skill. Apparently, pigeons do not suffer from this illusion, or at least they do not suffer from this illusion with this task, in which a near hit does not indicate that one getting closer to a win, because the outcomes are totally probabalistic.

The preference for near hit outcomes by humans may result from the large number of skill tasks in which humans tend to engage ( Stagner et al., 2015). It would be interesting to know if giving an animal a large number of skill tasks in which getting close to success represents a higher probability of success might alter an animal's avoidance of near-hit outcomes in tasks in which skill plays no role.

12. THE DEMOGRAPHICS OF SUBOPTIMAL CHOICE AND HUMAN GAMBLING

12.1 The Relation Between Level of Food Restriction and Suboptimal Choice.

Although humans often describe gambling as a form of entertainment, the fact that people with higher needs (those of lower socio-economic status) tend to gamble proportionally more than those with lower needs (those of higher status) (Lyk-Jensen, 2010; Worthington, 2001) suggests that this is not the primary motivation for gambling. If the pigeon model of suboptimal choice is an analog of human gambling, then one might expect the level of pigeons' food motivation be associated with their degree of suboptimal choice. Laude et al. (2012) found support for the relationship. They found that pigeons that were normally food restricted showed the typical suboptimal choice, whereas those that were minimally food restricted tended to choose optimally.

12.2 The Relation Between Impulsivity and Suboptimal Choice.

It has been proposed that the mechanism responsible for the suboptimal choice involved in gambling is likely to be impulsivity (Michalczuk et al., 2011; Nower and Blaszczynski, 2006). Impulsivity can defined as the inability to delay reinforcement and it has been assessed by way of delay discounting tasks in which an organism is given a choice between a small immediate reinforcement and a larger delayed reinforcement. The delay at which the organism is indifferent between the two alternatives defines the slope of the discounting function and the degree of impulsivity. Thus, impulsive individuals require that the delay to the larger amount of reinforcement be relatively short before they will prefer it and for them the slope of the discounting function would be relatively steep. We have recently found that the slope of the delay discounting function for pigeons is a good predictor of the degree to which they prefer the suboptimal choice in the gambling-like task ( Laude et al., 2014). Thus, impulsivity appears to be related to suboptimal choice for pigeons as well.

12.3 The Relation Between Housing and Suboptimal Choice.

There is some suggestion from research with rats that various extra-experimental environmental factors such as social and physical enrichment can affect a rat's propensity to self-administer drugs of addiction (Stairs & Bardo, 2009). Rats that are housed in an enriched group environment (a large cage with other rats and objects that are replaced regularly) show a significantly reduced tendency to self-administer drugs than rats that are normally (individually) housed. The mechanism responsible for the reduced self-administration of drugs by environmental enrichment has been hypothesized to be a reduction in impulsive behavior (Perry & Carroll, 2008) as well as the reduced effectiveness of conditioned reinforcers (Jones, Marsden, & Robbins, 1990). Impulsivity has also been implicated in human gambling behavior (Steel & Blaszczynski, 1998) and, as already noted, conditioned reinforcement has been proposed to account for suboptimal choice by animals (Dinsmoor, 1983). Furthermore, there is evidence that similar physiological mechanisms underlie compulsive gambling and drug addiction (Potenza, 2008).

In an attempt to determine the effect of housing conditions on suboptimal choice, we gave one group of pigeons experience in an enriched environment (a large cage with four other pigeons for 4 h a day), while the control pigeons remained in their normal one-to-a-cage housing (Pattison, Laude, & Zentall, 2013). When we exposed the pigeons from both groups to the gambling-like suboptimal choice task we found that the normally housed pigeons showed the typical suboptimal choice, whereas the enriched pigeons showed a significantly delayed choice of the suboptimal alternative. Thus, enriched housing, even for a relatively short 4-h a day, appears to retard the development of suboptimal choice. This finding has implications for the treatment of problem gambling behavior by humans. It implies that exposing human gamblers to an environment that is socially and physically enriched may reduce the attraction of gambling.

13. CONCLUSIONS

The suboptimal choice task provides a reasonable analog to human commercial gambling behavior. The mechanisms responsible for this behavior appear to be the relative lack of effectiveness of losses (nonreinforcement) in reducing the tendency to choose the suboptimal alternative, even when the losses occur on almost every trial (Smith & Zentall, in press; Stagner & Zentall, 2011; Vasconcelos et al., 2015). Furthermore, the relative probability of reinforcement associated with the two initial link alternatives appears to be relatively unimportant. Instead, the predictive value of the terminal link stimuli appears to be the primary determinant of initial link preference. Similarly, for most humans who gamble, it is the potential outcome for winning rather than the odds of winning that influences the tendency to gamble.

Interestingly, for pigeons, increasing the motivation to obtain reinforcement actually increases the tendency to choose suboptimally and thus obtain less food. Furthermore, providing pigeons with a somewhat enriched environment results in a significant reduction in the acquisition of suboptimal choice, a result that may have implications for the treatment of human problem gambling. Thus, the animal suboptimal choice task may provide a useful model to understand the mechanisms and treatment of human gambling behavior.

14. REFERENCES

Allais, M., 1953. Le comportement de l'homme rationnel devant le risque: critique des postulats et axiomes de l'école Américaine, Econometrica, 21, 503-546.         [ Links ]

Belke, T.W., Spetch, M. L., 1994. Choice between reliable and unreliable reinforcement alternatives revisited: Preference for unreliable reinforcement. Journal of the Experimental Analysis of Behavior, 62, 353-366.         [ Links ]

Breen, R.B., Zuckerman, M., 1999. 'Chasing' in gambling behavior: Personality and cognitive determinants. Personality & Individual Differences, 27, 1097-1111.         [ Links ]

Brunborg, G.S., Johnsen, B.J., Pallesen, S., Molde, H., Mentzoni, R.A., & Myrseth, H., 2010. The relationship between aversive conditioning and risk-avoidance in gambling. Journal of Gambling Studies, 26, 545-559.         [ Links ]

Crockford. D.N., Goodyear. B., Edwards. J., Quickfall. J., el-Guebaly. N., 2005. Cue-induced brain activity in pathological gamblers. Biological Psychiatry, 58, 787-795.         [ Links ]

Dinsmoor, J.A., 1983. Observing and conditioned reinforcement. Behavioral and Brain Science, 6, 693-728.         [ Links ]

Dinsmoor, J.A., 1985. The role of observing and attention in establishing stimulus control. Journal of the Experimental Analysis of Behavior, 43, 365-381.         [ Links ]

DSM-V American Psychiatric Association (2013). Diagnostic and Statistical Manual of Mental Disorders (5th ed.). Arlington, VA: American Psychiatric Association.         [ Links ]

Dunn, R., & Spetch, M. L. (1990). Choice with uncertain outcomes: Conditioned reinforcement effects. Journal of the Experimental Analysis of Behavior, 53, 201-218.         [ Links ]

Fantino, E., 1967. Preference for mixed- versus fixed-ratio schedules. Journal of the Experimental Analysis of Behavior, 10, 35-43.         [ Links ]

Fantino, E., 1969. Choice and rate of reinforcement. Journal of the Experimental Analysis of Behavior, 12, 723-730.         [ Links ]

Fantino, E., Abarca, N., 1985. Choice, optimal foraging, and the delay-reduction hypothesis. Behavioral and Brain Science, 8, 315-330.         [ Links ]

Fantino, E., Case, D.A. 1983. Human observing: Maintained by stimuli correlated with reinforcement but not extinction. Journal of the Experimental Analysis of Behavior, 40, 193-210.         [ Links ]

Fantino, E., Dunn, R., Meck, W., 1979. Percentage reinforcement and choice. Journal of the Experimental Analysis of Behavior, 32, 335-340.         [ Links ]

Fantino, E., Preston, R. A., & Dunn, R. (1993). Delay reduction: Current status. Journal of the Experimental Analysis of Behavior, 60, 159-169.         [ Links ]

Fantino, E., Silberberg, A., 2010. Revisiting the role of bad news in maintaining human observing behavior. Journal of the Experimental Analysis of Behavior, 93, 157-170.         [ Links ]

Field, M., & Cox, W. M., 2008. Attentional bias in addictive behaviors: A review of its development, causes, and consequences. Drug and Alcohol Dependence, 97, 1-20.         [ Links ]

Franken I. H. A., Stam C., Hendriks V. M., van den Brink, W., 2003. Neuropsychological evidence for abnormal cognitive processing of drug cues in heroin dependence. Psychopharmacology, 170, 205-212.         [ Links ]

Gipson, C. D., Alessandri, J.D., Miller, H.C., Zentall, T.R., 2009. Preference for 50% reinforcement over 75% reinforcement by pigeons. Learning & Behavior, 37, 289-298.         [ Links ]

Griffiths, M. (1999). Gambling technologies: Prospects for problem gambling. Journal of Gambling Studies, 15, 265-283.         [ Links ]

Hearst, E., Besley, S., Farthing, G.W., 1970. Inhibition and the stimulus control of operant behavior. Journal of the Experimental Analysis of Behavior, 14, 373-409.         [ Links ]

Holst R. J., van den Brink W., Veltman D.J., Goudriaan A.E., 2010. Why gamblers fail to win: A review of cognitive and neuroimaging findings in pathological gambling. Neuroscience and Biobehavioral Reviews, 34, 87-107.         [ Links ]

Hursh S.R., Fantino E., 1974 An appraisal of preference for multiple versus mixed schedules. Journal of the Experimental Analysis of Behavior, 22, 31-38.         [ Links ]

Jones G.H., Marsden C.A., Robbins T.W., 1990 Increased sensitivity to amphetamine and reward-related stimuli following social isolation in rats: possible disruption of dopamine-dependent mechanisms of the nucleus accumbens. Psychopharmacology, 3, 364-372.         [ Links ]

Kahneman, D., Tversky, A., 1979. Prospect theory: An analysis of decision under risk. Econometrica, 47, 263-291.         [ Links ]

Kendall, S. B. (1974). Preference for intermittent reinforcement. Journal of the Experimental Analysis of Behavior, 21, 463-473.         [ Links ]

Kendall, S. B. (1985). A further study of choice and percentage reinforcement. Behavioural Processes, 10, 399-413.         [ Links ]

Langer, E. J. (1975). The illusion of control. Journal of Personality and Social Psychology, 32, 311-328.         [ Links ]

Laude, J.R., Beckmann, J.S., Daniels, C.W., Zentall, T.R., (2014). Impulsivity affects suboptimal gambling-like choice by pigeons. Journal of Experimental Psychology: Animal Behavior Processes.40 (1), 2-11.         [ Links ]

Laude, J.R., Pattison, K.F., & Zentall, T.R., 2012. Hungry pigeons make suboptimal choices, less hungry pigeons do not. Psychonomic Bulletin & Review, 19, 884-891.         [ Links ]

Laude, J.R., Stagner, J.P., & Zentall, T.R. 2014. Suboptimal choice by pigeons may result from the diminishing effect of nonreinforcement. Journal of Experimental Psychology: Animal Behavior Processes, 40, 12-21.         [ Links ]

Lyk-Jensen, S.V., 2010. New evidence from the grey area: Danish results for at-risk gambling. Journal of Gambling Studies, 26, 455-467.         [ Links ]

MacLin, O. H., Dixon, M. R., Daugherty, D., & Small, S. L. (2007). Using a computer simulation of three slot gambling machines to investigate a gambler's preference among varying densities of near-miss alternatives. Behavioral Research Methods, 39, 237-241.         [ Links ]

Mazur, J.E. 1996. Choice with certain and uncertain reinforcers in an adjusting delay procedure. Journal of the Experimental Analysis of Behavior, 66, 63-73.         [ Links ]

McDevitt, M. A., Dunn, R. M., Spetch, M. L., & Ludvig, E. A. (2016). When good news leads to bad choices. Journal of the Experimental Analysis of Behavior, 105, 23-40.         [ Links ]

McDevitt, M. A., Spetch, M. L., & Dunn, R. (1997). Contiguity and conditioned reinforcement in probabilistic choice. Journal of the Experimental Analysis of Behavior, 68, 317-327.         [ Links ]

Michalczuk, R., Bowden-Jones, H., Verdejo-Garcia, A. Clark, L., 2011. Impulsivity and cognitive distortions in pathological gamblers attending the UK National Problem Gambling Clinic: a preliminary report. Psychological Medicine, 41, 2625-2635.         [ Links ]

Molet, M., Miller, H.C., Laude, J.R., Kirk, C., Manning, B., Zentall, T.R., 2012. Decision-making by humans as assessed by a choice task: Do humans, like pigeons, show suboptimal choice? Learning & Behavior, 40, 439-447.         [ Links ]

Nower, L. & Blaszczynski, A., 2006. Impulsivity and Pathological Gambling: A Descriptive Model. International Gambling Studies, 6, 61-75.         [ Links ]

Pattison, K. F., Laude, J. R., & Zentall, T. R. (2013). Social enrichment affects suboptimal, risky, gambling-like choice by pigeons. Animal Cognition, 16, 429-434.         [ Links ]

Perry J.L., Carroll M.E. 2008. The role of impulsive behavior in drug abuse. Psychopharmacology, 200, 1-26.         [ Links ]

Potenza M.N., 2008. The neurobiology of pathological gambling and drug addiction: an overview and new findings. Philosophical Transactions of the Royal Society: B, 363, 3181-3189.         [ Links ]

Pyke, G.H., Pulliam, H. R., & Charnov, E. L. (1977). Optimal foraging: A selective review of theory and tests. The Quarterly Review of Biology, 52, 137-154.         [ Links ]

Reid, R. L., 1986. The psychology of the near miss. Journal of gambling behavior, 2, 32-39.         [ Links ]

Rachlin, H, Green, L., 1972. Commitment, choice and self-control. Journal of the Experimental Analysis of Behavior, 17, 15-22.         [ Links ]

Roberts, W. A., 1972. Short-term memory in the pigeon: Effects of repetition and spacing. Journal of Experimental Psychology, 94, 74-83.         [ Links ]

Roper, K.L., Zentall, T.R., 1999. Observing behavior in pigeons: The effect of reinforcement probability and response cost using a symmetrical choice procedure. Learning and Motivation, 30, 201-220.         [ Links ]

Shafir, S., Reich, T., Tsur, E., Erev, I., & Lotem, A., 2008. Perceptual accuracy and conflicting effects of certainty on risk-taking behaviour. Nature, 453, 917-921.         [ Links ]

Smith, A. P., Bailey, A. R., Chow, J. J., Beckmann, J. S., & Zentall, T. R. (in press). Suboptimal choice in pigeons: Stimulus value predicts choice over frequencies. PLos ONE.         [ Links ]

Smith, A. P., & Zentall, T. R. (2016). Suboptimal choice in pigeons: Choice is primarily based on the value of the conditioned reinforcer rather than overall reinforcement rate. Journal of Experimental Psychology: Animal Behavior Processes, 42, 121-220.         [ Links ]

Spetch, M.L., Belke, T.W., Barnet, R.C., Dunn, R., Pierce, W.D. 1990. Suboptimal choice in a percentage-reinforcement procedure: Effects of signal condition and terminal link length. Journal of the Experimental Analysis of Behavior, 53, 219-234.         [ Links ]

Spetch, M.L., Mondloch, M.V., Belke, T.W., Dunn, R., 1994. Determinants of pigeons' choice between certain and probabilistic outcomes. Animal Learning & Behavior 22, 239-251.         [ Links ]

Stairs D.J., Bardo M.T., 2009. Neurobehavioral effects of environmental enrichment and drug abuse vulnerability. Pharmacology Biochemistry and Behavior, 92, 377-382.         [ Links ]

Stagner, J. P., Case, J. P., Sticklen M. F., Duncan, A. K., & Zentall, T. R., 2015. Do pigeons prefer alternatives that include near-hit outcomes? Journal of experimental psychology. Animal learning and cognition, 41, 247-254.         [ Links ]

Stagner, J.P., Laude, J.R., & Zentall, T.R., 2011. Sub-optimal choice in pigeons does not depend on avoidance of the stimulus associated with the absence of reinforcement. Learning and Motivation, 42, 282-287.         [ Links ]

Stagner, J.P., Laude, J.R., Zentall, T.R., 2012. Pigeons prefer discriminative stimuli independently of the overall probability of reinforcement and of the number of presentations of the conditioned reinforcer. Journal of Experimental Psychology: Animal Behavior Processes 38, 446-452.         [ Links ]

Stagner, J.P. Zentall, T.R., 2010. Suboptimal choice behavior by pigeons. Psychological Bulletin & Review, 17, 412-416.         [ Links ]

Steel Z., Blaszczynski A., 1998. Impulsivity, personality disorders and pathological gambling severity. Addiction, 93, 895-905.         [ Links ]

Stephens, D.W. Krebs, J.R. (1986). Foraging theory. Princeton, NJ: Princeton University Press.         [ Links ]

Tversky, A., Kahneman, D., (1974). Judgment under uncertainty: Heuristics and biases. Science, 185, 1124-1131.         [ Links ]

Vasconcelos, M., Monteiro, T., & Kacelnik, A. (2015). Irrational choice and the value of information. Scientific Reports, 5, 13874.         [ Links ]

Winstanley, C. A., Cocker, P. J. & Rogers, R. D. 2011. Dopamine modulates reward expectancy during performance of a slot machine task in rats: Evidence for a "Near-miss" effect. Neuropsychopharmacology, 36, 913-925.         [ Links ]

Worthington, A. C., 2001. Implicit Finance in Gambling Expenditures: Australian Evidence on Socioeconomic and Demographic Tax. Public Finance Review, 29, 326-342.         [ Links ]

Zentall, T. R., & Stagner, J. (2011a). Maladaptive choice behavior by pigeons: An animal analogue and possible mechanism for gambling (sub-optimal human decision-making behavior). Proceedings of the Royal Society B: Biological Sciences, 278, 1203-1208.         [ Links ]

Zentall, T. R., & Stagner, J. (2011b). Sub-optimal choice by pigeons: Failure to support the Allais paradox. Learning and Motivation, 42, 245-254.         [ Links ]