January 13, 2005 11:37 WSPC/167-FNL

00230

Fluctuation and Noise Letters

Vol. 4, No. 4 (2004) C13C20
c World Scientic Publishing Company

GAMBLERS PARADOX AND NOISE DRIVEN FLUX REVERSAL

IN KINETIC CYCLES: RESPONSE TO THE PRECEDING PAPER
BY PIOTROWSKI AND SLADKOWSKI

R. DEAN ASTUMIAN
Department of Physics and Astronomy
University of Maine
Orono, Maine 04469-5709, USA
astumian@maine.edu
Received 22 October 2004
Revised 9 November 2004
Accepted 9 November 2004
Communicated by Laszlo Kish

I give a simple analysis of the game that I previously published in Scientific American
which shows the paradoxical behavior whereby two losing games randomly combine to
form a winning game. The game, modeled on a random walk, requires only two states
and is described by a rst-order Markov process.
Keywords: Brownian ratchets; random games; brownian motors; random walk; uctuation induced transport.

1.

Introduction

In the mid 1980s my colleagues and I posed the following paradox [1, 2] in the
Proceedings of the National Academy of Sciences USA. Consider a random walk on
a cycle of bases governed by one of two sets of transition constants for stepping
between the bases. Using either set alone, the walk is biased to favor completion of
a counterclockwise cycle. However, periodic [1] or random [2] alternation between
the two sets causes net clockwise cycling!
The model is shown in Fig. 1. The transition coecients are given in terms
of three parameters: a kinetic bias, b, that controls the relative stability of states
2 and 4 vs. states 1 and 3; an output factor that controls the overall bias for
clockwise vs. counterclockwise cycling; and an external control parameter . For
= 1 the ratio of clockwise to counterclockwise transition coecients is unity
irrespective of the value of b or of , a condition often asserted to be necessary and
sucient for detailed balance [3] to hold. Nonetheless, oscillation or uctuation
of leads to net clockwise ux for b > 1 [1, 2]. A particularly interesting case
C13

January 13, 2005 11:37 WSPC/167-FNL

C14

00230

R. D. Astumian

results when randomly switches between two values, +A and A, with Poisson
distributed lifetimes with mean time 1 in each state, mimicking the behavior of
a non-equilibrium chemical reaction. This dichotomous noise induces clockwise
ux that persists for suciently large A even if < 1, a condition that biases the
ux through the cycle to be counterclockwise for xed with either = +A or
= A.

b e+

4
b

1/

3
+

b2 e

b2 e+
b e

Fig. 1. Four state kinetic model. The product of clockwise transition coecients divided by the
product of counterclockwise transition coecients is 2 irrespective of the value of b or of . With
< 1 a random walk on the four bases is biased such that completion of a counterclockwise cycle
is more likely than completion of a clockwise cycle for any xed value of . Nevertheless, when
b > 1, in the presence of dichotomic uctuation of between +A and A for suciently large A
(and large ) completion of a clockwise cycle is more likely than completion of a counterclockwise
cycle.

To illustrate as simply as possible this paradoxical behavior, I published [4] a very

simple game played with a checker stepping on part of a checkerboard. The stepping
is decided by the roll of a pair of dice according to one of two sets of rules. With
either set of rules the player loses more than he/she wins. Random alternation
between two sets of rules for the stepping, achieved by ipping a coin to decide
which set of rules is to be used for each roll, leads to more wins than losses.
In the preceding paper [5] in this issue Piotrowski and Sladkowski (P &S henceforward) discuss this Gamblers paradox and assert that my analysis was awed,
and that the game does not display the paradoxical behavior that I claimed. Here
I show that P &S consider only a very special case for my game dierent than the
case treated in my Scientic American article [4], and that in general my game does
show the paradoxical behavior rst demonstrated in Refs. 1 and 2. The dierence
in our conclusions arises from a subtle but critical distinction between frequencies
and probabilities.
2.

Astumian Game

The game is shown in Fig. 2. Game 1 is played as follows the checker starts on
the middle black square (state 3). The player rolls a pair of standard six-sided dice
and follows the instructions in Table 1. Since the checker starts on a black square,

January 13, 2005 11:37 WSPC/167-FNL

00230

Gamblers Paradox

5
win

lose

Table 1
1(H)

C15

Table 2

white

black

white

2(T)

black

7, 11 (W = 8/36)

2,3,12 (W = 4/36)

2,4,12 (B =5/36)

2,4,12 (W =5/36) 2,3,12 ( = 4/36)

B

other (24/36)

other (29/36)

other (29/36)

11 (B = 2/36)

11 (W = 2/36)

7, 11 (B = 8/36)

other ( 24/36)

Kinetic Diagram for the Astumian Game

B

W,i

W,i

B,i

B,i

W,i

W,i

Kinetic Diagram for the P&S version

pW,i

1-pW,i W
pW,i =

1-pB,i
W,i

W,i+ W,i

pB,i
1-pW,i
pB,i =

pW,i

B,i
B,i+ B,i

Fig. 2. The Simplest Astumian Game is played with a checker on ve squares of a checker board.
The stepping of the checker towards a win or loss is governed by the roll of a pair of dice
according to one of the two rule tables labelled Table1 and Table 2. In the randomly combined
game a coin is ipped before each roll. If the coin lands heads the following roll is played according
to Table 1, and if it lands tails the following roll is played according to Table 2. The game can be
represented as a kinetic diagram. P &S transformed the game to one in which the checker steps
left or right on each iteration, resulting in a subtly dierent kinetic diagram used as the basis for
their preceeding paper [5].

on a roll of 11, the checker is displaced one square to the right (R); on a roll of 2,4,
or 12, the checker is displaced to the left (L); on any other roll (3,5,6,7,8,9,10) the
checker is left in place (N). The frequency for each possibility, based on the number
of ways to make the given rolls out of 36 possible rolls, is given in parentheses
R = 2/36; L = 5/36; N = 29/36. When the checker is on a white square the
instructions from Table 1 in the column below white are followed. The player
loses if the checker reaches the left hand black square marked lose (state 1), and the
player wins if the checker reaches the right hand black square marked win (state 5).
Game 2 is played in exactly the same way except that the instructions in Table 2
govern the displacement of the checker. A move in either game consists of rolling
the dice and displacing the checker (R,L, or N) according to the instructions in the
appropriate table.
Game one can be represented by the simple kinetic diagram for jumping between
the ve dierent states:

January 13, 2005 11:37 WSPC/167-FNL

C16

00230

R. D. Astumian

4
36

8
36
5
36

2
36
4
36

8
36

where the numbers written above and below the arrows are the frequencies of transitions between neighboring states. The player wins if he/she winds up at state ve,
and loses if he/she reaches state 1. If the player starts from state 3 the ratio of the
probability of losing to the probability of winning is 5 4/(8 2) = 20/16 = 5/4 as
reported by P &S as well as by me [4]. The same numerical result for the ratio of
losses to wins is obtained for the game 2, which can be represented by the kinetic
diagram below

5
36

2
36
4
36

8
36
5
36

2
36

5.

Both games result in more losses than wins. If however before each roll of the dice
(i.e., after each move) a coin is ipped to determine whether the next roll is played
according to the rules in Table 1 (on heads) or according to the rules in Table 2 (on
tails) the situation is very dierent.
To calculate the result of combining the two games by a coin ip we average the
frequencies. This can be seen very simply the coin ip randomizes the chance
of playing by rule set 1 or rule set 2 for each roll of the dice. Starting at the
middle there is a 50/50 chance of playing by rule set 1, for which the frequency of
displacement to the right is 2/36, and a 50/50 chance of playing by rule set 2 for
which the frequency of displacement to the right is 8/36. Thus the net frequency
of being displaced to the right on the next roll is 10/72. Exactly the same 10/72
holds for the frequency of displacement to the right from a white square. The
average frequency for displacement to the left from either white or black is 9/72.
The diagram for the combined game, where the frequencies are the arithmetic mean
of the frequencies for the above two games, is shown below:

9
72

10
72
9
72

10
72
9
72

10
72

5.

Clearly the combined game is one which on average is winning (10 10/(9 9)
equivalent to 55% wins expected) even though either game alone is losing (8
2/(5 4) equivalent to 45% wins expected). This conrms my analysis published
in Scientic American the transitions from states 2,3, and 4 to the right occur
with greater frequency than do those to the left, resulting in more wins than losses.
Why does the simplication of my game studied by P &S in the preceeding paper
not give rise to the paradox?
3.

Frequency vs. Probability

The mistaken conclusions of P &S apparently results from their misunderstanding of

the dierence between frequency and probability. In coming up with their diagrams

January 13, 2005 11:37 WSPC/167-FNL

00230

Gamblers Paradox

C17

P &S make a critical error without justication they converted the frequencies
i , i to probabilities pi = i /(i + i ), qi = i /(i + i ) (i = W, B) such that
pi + qi = 1. For game 1 this results in the following diagram shown in the paper by
P &S
1

1
3

2
3
5
7

2
7
1
3

2
3

5.

The corresponding diagram for game 2 from P &S is

1

5
7

2
7
1
3

2
3
5
7

2
7

5.

Both of these games are losing by the same percentage as Games 1 and 2 above,
where we calculate the ratio of losses to wins as the product qW qB /(pW pB ).
Now if we combine these by taking the arithmetic mean of the probabilities we get
(as reported by P &S)

11
21

10
21
11
21

10
21
11
21

10
21

5.

What happened the paradox has disappeared! Clearly the combined game is now
a losing proposition just like games one and two. By converting from frequencies
to probabilities information about scale (number of rolls spent on a given square)
is lost the frequency picture yields one and only one probability picture, but
obviously the probability picture could have resulted from any number of frequency
pictures, e.g., do we divide by 36, or by 360, etc. When we play just a single simple
game with only one scale, the discarded information does not change the ratio of
losses to wins. However, the situation is dramatically dierent when we combine
games 1 and 2 by ipping a coin after each roll of the dice to select the rule set by
which the next decision is made. Now there are two scales in the problem, 1/36 from
the dice, and 1/2 from the coin ip, so we cannot blithely ignore the normalization
factor. This is also manifest in the original formulation of the paradox the
direction of ux in the four state cycle depends on the frequency of transition
between +A and A, , relative to the frequency for stepping on the four bases [2].
As I explicitly noted in my Scientic American paper The game has an asymmetry: according to the rst set of rules the piece tends to spend a larger number
of rolls on the black square than on a white one, and vice versa for the second set of
rules. The coin ip erases this asymmetry. [4] The interpretation of P &S, where
the checker remains on any square for only one move, eliminates this key feature.
4.

General Discussion of the Role of Lifetime

Referring to the kinetic diagram for the Astumian game on Fig. 2 we see that the
general condition for obtaining the paradoxical result that combining two losing

January 13, 2005 11:37 WSPC/167-FNL

C18

00230

R. D. Astumian

games yields a winning game can be written

W,1 B,1 < W,1 B,1
W,2 B,2 < W,2 B,2

(1a)
(1b)

(W,1 + W,2 )(B,1 + B,2 ) > (W,1 + W,2 )(B,1 + B,2 ).

(1c)

This is very easily met in general, for example with the values from my Scientic
American article [4] given in the tables in Fig. 2. If however we constrain the
lifetimes in the Black and White states to be equal ((W,i +W,i ) = (B,i +B,i ), i =
1, 2) Eqs. 1 reduce to W,1 < B,1 , W,2 < B,2 and (W,1 + W,2 ) > (B,1 + B,2 )
which is a contradiction. The Paradox cannot occur when the lifetimes in the two
states are equal. This is what P &S have unwittingly insisted on in their reduction
of the Astumian game. Explicitly, the requirement for their version of my game can
be written
pW,1 pB,1 < (1 pW,1 )(1 pB,1 )
pW,2 pB,2 < (1 pW,2 )(1 pB,2 )
(pW,1 + pW,2 )(pB,1 + pB,2 ) > (2 pW,1 pW,2 )(2 pB,1 pB,2 ).

(2a)
(2b)
(2c)

Eqs. (2) reduce to (pW,1 + pB,1 ) < 1, (pW,2 + pB,2 ) < 1, and (pW,1 + pB,1 + pW,2 +
pB,2 ) > 2, a contradiction. This contradiction shows that the paradox cannot occur
in the P &S version of the Astumian game. It is worth noting that Eq. 1 is the
mathematical requirement for a widely celebrated paradox Simpsons paradox [6],
described in many textbooks [7] (see also [8]).
5.

Parrondo Game

In 1996, Juan Parrondo, in a lecture entitled Eciency of Brownian Motors (see

the web site http://seneca.s.ucm.es/parr/) introduced games based on tossing one
of several biased coins, where two losing games combine to form a winning game.
This was the rst discussion of Brownian motors in the context of games per
se. Parrondos games were inspired by ashing ratchets [9, 10], a simple model
for brownian motor mechanisms [11, 12] often used to model biomolecular motors.
Because his games are played with a coin, each ip of the coin results in a binary
decision: win-lose, or left-right. To achieve the required asymmetry, one of the
games Parrondo described involved a rather complicated rule where the coin to be
ipped is determined by the amount of money the player has on that turn. However,
Parrondos coin ip games can be mapped onto a Markov cycle [13, 14] identical
to the cycles used in the original formulation of the paradox [1, 2] except that the
sum of the two transition rates out of each state must equal unity in the Parrondo
formulation. As shown by Parrondo and colleagues [13], and more recently by Lee
et al. [14] and Iyengar and Kohli [15], with this constraint the minimal model for
demonstrating the paradoxical reversal of the direction of a ow requires a secondorder Markov (three-state or modulo 3) system.
In contrast, the Astumian game, which is the discrete-time analogue of the
continuous time two-state chemical kinetic mechanism [16] for a Brownian motor
[12], clearly demonstrates that a rst-order Markov chain a two-state (modulo2) system is sucient to demonstrate the paradoxical behavior that two losing

January 13, 2005 11:37 WSPC/167-FNL

00230

Gamblers Paradox

C19

games can be combined to form a winning game when the marker can remain on the
same square (or the players capital can have the same value) for several iterations.
6.

Conclusion

The importance of simple models such as Astumian games and Parrondo games has
to do with their relevance to physical systems [8]. The underlying mechanism for
transition between states in kinetic cycles such as that in Fig. 1 is thermal activation
that results from the brownian motion of molecules in the system. Extensions and
generalization of this model have led to the ourishing eld involved in studying
Brownian motors molecules and systems that combine directionless input energy
with thermal (or other) noise to lead to directed transport [11,12]. In the Astumian
game, The dice play the role of thermal noise, allowing occasional transition from
one square to the next. The net tendency to lose reects the eect of external
forces that cause the motor to lose energy. The coin toss mimics the input of
energy from the non-equilibrium chemical reaction that powers the motor. The
reversal from losing to winning addresses what I have always considered one
of the deepest paradoxes associated with molecular motors and pumps. In any
chemical state, the motor tends to move in the direction driven by any externally
applied force. If the motor, powered by a non-equilibrium reaction, cycles through
dierent chemical states, however, the direction can be reversed, and the motor can
move energetically uphill. The Astumian games show how randomly interleaving
stochastic events of game 1 (one chemical state of the motor) with those of game
2 (a dierent chemical state of the motor) can reverse the predicted outcome of
playing game 1 or game 2 alone. It is important to note that this reversal is a
purely statistical result there need be no direct mechanical interaction between
the coin and the checker (or more subtly, between a biomolecular motor and the
chemical reaction, e.g. ATP hydrolysis, that provides the energy for switching)
other than that required to change the kinetic rules [16].
References
[1] H. V. Westerho, T. Y. Tsong, P. B. Chock, Y. D. Chen and R. D. Astumian, How
enzymes can capture and transmit free energy from an oscillating electric eld, Proc.
Natl. Acad. Sci. USA 83 (1986) 47344737.
[2] R. D. Astumian, P. B. Chock, T. Y. Tsong, Y. D. Chen and H. V. Westerho, Can
free energy be transduced from electric noise? Proc. Natl. Acad. Sci. USA 84 (1987)
434438.
[3] L. Onsager, Reciprocal relations in irreversible processes. I., Phys. Rev. (1931) 37
405426.
[4] R. D. Astumian, Making molecules into motors, Sci. Am. 285 (7) (2001) 5664.
[5] E. W. Piotrowski and J. Sladkowski, On the applicability of Astumians model in
describing Parrondo eects, Fluct. Noise Letts. (2004) preceeding paper.
[6] E. H. Simpson, The interpretation of interaction in contingency tables, J. Roy. Stat.
Soc. B 13 (1951) 238241.
[7] D. Moore and G. McCabe, Introduction to the Practice of Statistics, W. H. Freeman
and Company, New York (1993).
[8] R. D. Astumian, Paradoxical games and a minimal model for a brownian motor, in
press, Am. Journ. Phys. (2004).

January 13, 2005 11:37 WSPC/167-FNL

C20

00230

R. D. Astumian

[9] R. D. Astumian and M. Bier, Fluctuation driven ratchets: molecular motors, Phys.
Rev. Lett. 72 (1994) 17661769.
[10] J. Prost, J. F. Chawin, L. Peliti and A. Ajdari, Asymmetric pumping of particles,
Phys. Rev. Lett. 72 (1994) 26522655.
[11] P. Reimann, Brownian motors: Noisy transport far from equilibrium, Phys. Rep. 361
(2002) 57265.
[12] R. D. Astumian and P. Hanggi, Brownian motors, Phys. Today (2002).
[13] J. M. R. Parrondo, G. Harmer and D. Abbot, New paradoxical games based on Brownian ratchets, Phys. Rev. Lett. 85 (2000) 52265229.
[14] Y. Lee, A. Allison, D. Abbott and H. E. Stanley, A minimal Brownian ratchet, an
exactly solvable model, Phys. Rev. Lett. 91 (2003) 220601xx.
[15] R. Iyengar and R. Kohli, Why Parrondos paradox is irrelevant for utility theory, stock
buying, and the emergence of life, Complexity 9 (2003) 2327.
[16] R. D. Astumian, P. B. Chock, T. Y. Tsong and H. V. Westerho, Eects of oscillations
and energy driven uctuations on the dynamics of enzyme catalysis and free-energy
transduction, Phys. Rev. A 39 (1989) 64166435.