SSRN Id2472783 PDF

Electronic copy available at: http://ssrn.
com/abstract=2472783

PAIRWISE COMPARISON OF CHESS OPENING VARIATIONS

JAMAL MUNSHI

ABSTRACT: It is demonstrated that a methodology previously described for comparing a number of different test openings with
a neutral and perfect opening may also be used to make pairwise comparisons of alternate continuation lines of any chess
opening. Such comparisons show, for example, that Accelerated Dragon variations initiated by black are not robust because they
depend on white's cooperation. Rather than these specific findings, however, the real purpose of this paper is to further refine,
validate, and demonstrate an objective methodology for evaluating the relative merit of chess openings, a subject of great interest
to chess players at all levels of play.
1
.

1. INTRODUCTION

This paper is the fourth of a series in a study undertaken to develop a generally applicable methodology
for the objective evaluation of chess openings. The proposed methodology uses controlled experiments
with chess engines to compare chess openings. In a previous paper, a dual engine experimental design
(DED) was used to evaluate a selected set of queen's pawn openings (Munshi, Comparing Chess
Openings Part 3, 2014). The dual engine test is more comprehensive than single engine analysis because a
greater number of opening variations are explored and the test is therefore more thorough (Munshi,
Comparing Chess Openings, 2014).

In this paper we describe and demonstrate a further application of the DED methodology by showing that
it may be used to make pairwise comparisons
2
of two alternative continuation lines of any chess opening
without the need to identify a neutral and perfect opening as a standard. For example, in the Yugoslav
Attack should white play 9.Bc4 or castle queenside instead? In all such cases, pairwise comparisons may
be used to determine whether one of the two variations being compared offers a statistically higher
probability of winning ceteris paribus.

The motivation for this study is that conventional methods of evaluating chess openings are inadequate
and unreliable (Munshi, A Method for Comparing Chess Openings, 2014). As a result, there are
conflicting opinions on the relative merit of different lines in the opening book and these opinions have
engendered ongoing debates that have no satisfactory resolution. The specific findings of this paper with
respect to the variations compared may not be relevant to chess games in which the outcome is
determined primarily by move errors, as may be the case in most chess games. Yet, there is a great deal of
interest among chess players at all levels of play in the relative merit of alternate opening lines
3
. The
value of this work, therefore, lies not so much in the findings themselves but in offering an objective and
quantitative methodology for comparing chess openings to those who wish to make such comparisons.

1
Date: July 2014
Key words and phrases: chess openings, chess engines, refutation, methodology, Monte Carlo simulation, numerical methods,
probability vector, Euclidean distance, robust statistics, bootstrap, first move advantage, chess
Author affiliation: Professor Emeritus, Sonoma State University, Rohnert Park, CA, 94928, munshi@sonoma.edu
2
There is a Wikipedia document that explains the concept of pairwise comparison in statistics (Wikipedia, 2014).
3
Consider, for example, the number of books, websites, and Youtube videos devoted to this topic.
Electronic copy available at: http://ssrn.com/abstract=2472783
PAIRWISE COMPARISON OF CHESS OPENING VARIATIONS, JAMAL MUNSHI, 2014 2

2. THEORY

Chess games may be thought of as a stochastic trinomial process driven by an unknown and unobservable
underlying probability vector given by

Equation 1 = [p
w
, p
b
, p
d
]

where is a vector with two degrees of freedom, p
w
is the probability that white will win, p
b
is the
probability that black will win, and p
d
= 1-p
w
-p
b
is the probability that the game will end in draw. So for
example if =[0.15,0.10,0.75] then each chess game played under these conditions is like the toss of a
three-sided "coin" that has a 15% probability of coming up "white wins", a 10% chance of coming up
"black wins", and a 75% probability of coming up "draw". Similarly, if 300 of these coins are tossed we
would expect to see something close to but not exactly the number of white wins 45+/-, the number of
black wins 30+/-, and the number of draws225+/-. The sample data will vary from one toss to the next.

The components of the probability vector are determined by (1) white's first move advantage or FMA,
(2) the general rate of imperfection in the moves or IMP, (3) the difference in playing strength between
the player making white moves and the player making black moves, or DIFF, and (4) an effect of the
opening employed or OE (Munshi, A Method for Comparing Chess Openings, 2014).

FMA is the probability that white will win by virtue of its first move advantage. It derives from the rule
that white moves first and is therefore one move ahead of black for the rest of the game. The value of
FMA is a universal constant . Its estimated value is FMA0.0589+/-
4
. This means that in every chess
game before the first move has been played there is approximately a 5.89% probability that white will win
simply because white will move first.

IMP is the probability that the game will end in a win by virtue of move imperfections. The value of IMP
depends on the level of play and in our experiments it is estimated to be IMP0.02+/-
5
. This means that
there is approximately a 2% probability that a game that would otherwise have ended in draw will
become decisive because of move imperfections. Such imperfections take the form of opportunities
missed, threats undetected, or traps not foreseen (Avni, 2011). The higher the level of play, the lower the
value of IMP
6
.

DIFF is the probability that a game will be won by virtue of a difference in playing strength. The presence
of DIFF causes games that might otherwise have ended in draw to be won by the stronger player. Playing
strength differential is normally measured on the so called Elo scale
7
.

4
An estimate of white's first move advantage is provided in the Appendix.
5
An estimate of IMP in our engine experiments is provided in the Appendix.
6
An estimate of IMP for titled players is provided in the Appendix.
7
Since the probability vector has two degrees of freedom, playing strength is a two dimensional entity. The attempt to assess
playing strength in terms of a single scalar measure such as the Elo rating is therefore flawed and the subject of a forthcoming
paper.

OE is the probability that the game will end in a win for one of the colors because of the opening
employed. In this study we take it to be axiomatic that there exist neutral and perfect openings with a
value of the opening effect of OE=0. For example the Sicilian Defense Najdorf Variation is widely
considered to be such an opening (DeFirmian, 2008).

Neutral and perfect openings are useful for measuring values of IMP and DIFF because OE does not
interfere in game outcomes. Openings that are not neutral and perfect may be referred to as innovations
where the innovator attempts to change the probability vector in its favor. In previous papers in the study
of chess openings we found that the innovations that do change the probability vector do so in favor of the
opponent and to the detriment of the innovator. For example, the King's Gambit innovation by black is
found to change the probability vector against black by offering white an additional 9.5% probability of
winning (Munshi, A method for comparing chess openings, 2014).

Opening effects cannot be measured in the presence of large values of DIFF and IMP. Experiments
designed to measure the effect of the opening must therefore minimize the values of IMP and DIFF so
that the opening effect can be observed. Statistically, on the average, white will win half of the games
made decisive by IMP and black will win the other half. When DIFF = 0 and the opening is perfect and
neutral, white wins by virtue of FMA and IMP but black wins only by virtue of IMP. Under these
conditions, the percentage win by black is an estimate of IMP/2 and the percentage win by white is an
estimate of FMA+IMP/2. For details please refer to the first paper in this series (Munshi, A Method for
Comparing Chess Openings, 2014)
8
.

When the opening is not perfect and neutral, it may change the probability vector in favor of one side.
This idea forms the theoretical foundation of the proposed methodology. Our hypothesis is that the choice
of opening line played can change and that therefore chess engine experiments under controlled
conditions may be used to detect the effect of openings on the probability vector .

In pairwise comparisons of two variations of an opening, the existence of a neutral and perfect opening
becomes less important. In this case we postulate only that in controlled experiments with all other
variables held constant and with DIFF and IMP minimized and held close to zero, if the same underlying
probability vector could have generated the observed sample statistics for both variations then the choice
of variation cannot be based on the probability of winning. The choice is then made on strategic and
subjective factors.

If that is not the case then the probability vectors implied by the two variations are different. If so, we
must determine whether the difference in the probability vectors favors one side or whether it simply
changes the probability of decisive games in a neutral direction without an advantage to either side. It is
only in the former case that the choice of variation becomes an important decision for chess players in
terms of the probability vector.

8
Some example computations are included in the Appendix.

In comparing two variations, variation-1 and variation-2, our research question and hypotheses are set up
as follows:

Equation 2 Research question: Is
1
=
2
?
Equation 3 Null hypothesis: H
0
:
1
=
2

Equation 4 Alternate hypothesis H
a
:
1
2

If the true and unknown component probabilities p
w
and p
b
could be plotted in Cartesian coordinates and
the Euclidean distance between variation-1 and variation -2 computed and designated as , then
1
2

implies >0. Therefore, we may write the testable hypotheses as:

Equation 5 H
0
: =0
Equation 6 H
a
: >0
Since we can't observe , we take a sample from each variation and measure the Euclidean distance d
between the two opening variations to be compared. We can then use the mean and variance of d to test
Equation 5. If we fail to reject H
0
in the test, we immediately reach the conclusion that the evidence does
not indicate that the probability vector is different in the two variations being compared and so we have
to accept the possibility that the same underlying probability vector generated both sets of sample data. In
other words we cannot reject the hypothesis that
1
=
2
.

If we reject H
0
, however, we know that the probability vectors are different, that
1
2
, but we still don't
know the direction of the change. Further tests are necessary to determine whether the change favors
white, whether it favors black, or whether the change is in a neutral direction and favors neither white nor
black. Only if we reject H
0
and we find that the distance vector is in a direction that implies a difference
in relative advantage can we propose that the choice of variation can be made on the basis of probability
of winning.

In cases where we have rejected H
0
and we know that >0 but the distance vector lies in a neutral
direction, the choice of variation may be based on a preference for decisive versus drawish
9
games. If the
distance vector pointing to variation-x lies in the first quadrant then variation-x offers a higher probability
of decisive games; and if it lies in the third quadrant then variation-x offers a greater probability of draw.

If the distance vector lies in second or fourth quadrants then and we have rejected H
0
we may infer that
the two variations differ with respect to relative advantage. The second quadrant offers an advantage to
black and the fourth quadrant offers an advantage to white.

9
Chess players often use the term "drawish" for opening strategies that are more likely to end in draw.

3. METHODOLOGY
Well recognized and established opening book databases (Meyer-Kahlen, 2000) (Jones & Powell, 2014)
are used to select four variations pairs for comparison. The selected variations and their move sequences
are shown in Table 1. Each variation pair represents an opening strategy decision between two
alternatives.

In Decision #1, black must decide whether to play 5.a6 or to initiate the Dragon variation with 5.g6. In
Decision#2, black wishes to play the Accelerated Dragon and expects white to play 3. d4, and we
compare that line with the mainline continuation
10
if white plays 3.Bb5 instead of 3. d4. Similarly, in
Decision#3 black wishes to play the Hyper Accelerated Dragon and again expects white to play 3.d4 and
we compare that line with the continuation initiated by 3. c3 instead of 3. d4. In Decision #4, white must
decide whether to play 9.Bc4 or whether to castle queenside with9.O-O-O.

If we knew the probability vector = [p
w
, p
b
, p
d
] at the end of each of the move sequences to be
compared we could make the decision very easily. White would choose the variation with higher
p
w
or lower p
b
or both and black would choose the variation with higher p
b
or lower p
w
or both.
In cases where the probability vector is the same in the two variations or in cases where p
w
and
p
b
have changed in the same direction and in the same proportion, the probability of win cannot
be used as the decision criterion. In such cases, the decision would be based on other criteria
such as personal preference or playing style but not on the probability of winning.

Pair Variation pair to be compared Move sequence Decision
1 B90 Najdorf Variation 1.e4c5 2.Nf3d6 3.d4cxd4 4.Nxd4Nf6 5.Nc3a6 5a6
B70 Dragon Variation 1.e4c5 2.Nf3d6 3.d4cxd4 4.Nxd4Nf6 5.Nc3g6 5g6
2 B34 Accelerated Dragon 1.e4c5 2.Nf3Nc6 3.d4cxd4 4.Nxd4g6 5.Nc3Bg7 3.d4
B31 Accelerated Dragon/Rossolimo 1.e4c5 2.Nf3Nc6 3.Bb5g6 4.Bxc6dxc6 5.d3Bg7 6.h3 3.Bb5
3 B34 Hyper Accelerated Dragon 1.e4c5 2.Nf3g6 3.d4cxd4 4.Nxd4Nc6 5.Nc3Bg7 3.d4
B27 Hyper Accelerated Dagon 1.e4c5 2.Nf3g6 3.c3Bg7 4.d4 5.cxd4cxd4 3.c3
4 B76 Modern Yugoslav 1.e4c5 2.Nf3d6 3.d4cxd4 4.Nxd4Nf6 5.Nc3g6 6.Be3Bg7 7.f3Nc6 8.Qd2O-O 9.O-O-Od5 9.O-O-O
B77 Classical Yugoslav 1.e4c5 2.Nf3d6 3.d4cxd4 4.Nxd4Nf6 5.Nc3g6 6.Be3Bg7 7.f3Nc6 8.Qd2O-O 9.Bc4Bd7 9.Bc4
Table 1 Variation pairs compared in this study

However, we don't know the value of the probability vector for any of the variations because the
probability vector is unobservable. To proceed we must use the statistical tools of estimation and
probability. First we estimate the value of the probability vector for each of the two variations to be
compared. To do that we use a controlled engine experiment using a sample size of n=300 games. Two
chess engines are used. The engines chosen are Houdini3 Pro and Houdini4 Pro (Houdart, Houdini Chess,
2013). In the experiment each engine plays 150 games as white and 150 games as black. The engine
search depth is fixed at 21 plies. All other engine parameters are set to their default values. There is one
engine experiment for each variation with the initial move sequence fixed according to Table 1. Engine
computations begin when the fixed move sequence in Table 1 is exhausted. The Deep Shredder (Meyer-
Kahlen, Deep Shredder, 2007) GUI
11
is used as the platform that conducts the engine experiments.

10
The reference lines used are consistent with the chesstempo.com opening book (Jones & Powell, 2014) set to Elo of 2600+.
11

Graphical User Interface

The results of each experiment are used to estimate the value of the probability vector for the
corresponding variation. For example, if in an experiment we find that out of 300 games, white won
45games and black won 15 games, we estimate that p
w
=45/300 = 15%, p
b
=15/300 = 5%, and p
d
=1-p
w
-p
b
=
80%. Therefore our estimation of the probability vector is =[0.15,0.05,0.80] or simply as =[45,15,240]
as long as we understand that division by 300 is implied. We still don't know what the value of really is
but this vector serves as our best unbiased estimate.

Once we have the estimated values of for the two opening variations to be compared we can estimate
how different they are by computing the Euclidean distance between them in two dimensional
12
Cartesian
x-y space with x=number of wins by white and y=number of wins by black. For example, the squared
Euclidean distance between
1
=[45,15,240] and
2
=[10,25,265] is d
2
= (45-10)
2
+ (15-25)
2
= 1325. The
distance is the square root of the squared distance so that d=36.4. We use the letter d to represent the
observed distance in the sample and the Greek letter for the actual and unobservable distance in the
population from which the sample of 300 games was taken.

The observed distance d is an estimation. We are unsure about its value because the estimated values of
used in its computation are not known with certainty. To estimate the degree of uncertainty in the value of
d we use a Monte Carlo procedure described in an earlier paper (Munshi, A Method for Comparing Chess
Openings, 2014) to generate one thousand simulated repetitions of the two experiments and compute one
thousand corresponding distances. These distances serve as a simulated sampling distribution of the
observed sample distance and the standard deviation of these distances serves as the standard deviation of
the sampling distribution of distances.

We can now carry out the hypothesis test for the real and unobservable distance according to Equations
5 and 6 by using a t-test to compute the probability that we would observe a sample distance as large or
larger if the value of is zero. We set our threshold of disbelief at =0.001 as recent studies have shown
that higher values of risk an unacceptable degree of irreproducibility (Johnson, 2013). If the computed
probability from the t-test is less than we reject H
0
and conclude that > 0 because we don't believe that
this sample could have been drawn from a population in which =0.

The distance is a vector in two dimensional space. Both its magnitude and its direction are important in
making a comparison between two variations. If > 0 it means that the probability vectors of the two
variations are different but that does not necessarily imply a relative advantage of one over the other. For
example, if
1
=[0.10,0.05,0.85] and
2
=[0.20,0.10,0.70] there is no difference in the relative advantage
and so we would consider this change to be in a neutral direction.

This means that not only the magnitude of the distance but also its direction is important in the
comparison. In the Cartesian x-y coordinate system with x=number of white wins and y=number of black
wins, the first and third quadrants may be thought of as neutral directions and the second and fourth
quadrants as being directions in which the relative advantage is different (Munshi, Comparing Chess
Openings Part 3, 2014). The second quadrant holds an advantage for black and the fourth quadrant holds
an advantage for white.

12 Although the vector has three components it has only two degrees of freedom because the number of draws is determined once we know the number of games won by white and the number of
games won by black.

For each pairwise comparison we use a combination of the hypothesis test and the directional test to
determine if the decision about which variation to play can be made on the basis of the probability of
winning ceteris paribus. If we fail to reject H
0
the observed distance is not large enough and so could be
zero. We can therefore immediately conclude that the data do not indicate that the choice of variation is
important from point of view of the probability vector. In that case the choice of variation may be made
on a subjective basis such as personality and style of play, or perhaps on advance information about the
opponent's opening skills.

If we reject H
0
we must check the direction of the vector. If the direction is in a neutral direction then the
choice may be important in terms of whether one wishes to play a line that leads to more draws or one
that leads to more decisive games. However, if we reject H
0
and the direction is in a non-neutral direction,
then the probability of winning, ceteris paribus, will depend on the choice of variation.

4.0 DATA ANALYSIS

4.1 Decision #1: Sicilian Defense Najdorf (5a6) versus the Dragon (5g6).
As shown in Table 1, we assume that the first five moves by white and the first four moves by black have
been played as 1.e4c5 2.Nf3d6 3.d4cxd4 4.Nxd4Nf6 5.Nc3 and that black must now decide whether to
play 5a6 (Najdorf) or 5g6 (Dragon). The results of the engine experiments for these two alternatives
are shown in Table 2 and depicted graphically in Figure 1. All six hundred games played in the two
experiments are available in PGN format in the online data archive for this paper (Munshi, Variations:
PGN Files, 2014). The rarity data are derived from opening book statistics reported by Jones and Powell
(Jones & Powell, 2014) with the level of play set to Elo > 2500 and is computed relative to the more
popular Najdorf Variation. It shows that the Najdorf Variation is preferred by a ratio of more than six to
one.

Variation Decision Rarity Games played Won by white Won by black Draw
B90 Najdorf Variation 5a6 1.00 300 20 4 276
B70 Dragon Variation 5g6 6.64 300 23 2 275
Table 2 Sample data for the Sicilian Najdorf and Dragon variations

Figure 1 Graphical depiction of the observed distance between the Najdorf and Dragon variations
The Euclidean distance between the two variations may be visualized as the length of the line in Figure 1.
The value of the observed distance d and its test against H
0
: =0 are summarized in Table 3.
5...a6
5...g6
0
1
2
3
4
5
19 20 21 22 23 24
W
o
n

b
y

B
l
a
c
k

Won by White
Sicilian Najdorf vs Dragon

The hypothesis test may be visualized in the graphical depiction of the simulated repetitions of the
experiment in Figure 2. The computational details of the simulated sampling distribution are available in
the online data archive for this paper (Munshi, Variations: Simulation files, 2014).

Hypothesis test = 0.001
H
0
=0
H
a
>0
Observed distance d=3.606
Standard deviation =4.903
t-value t=0.735
p-value p=0.4622
Since p> Fail to reject
Table 3 Hypothesis test: Sicilian Najdorf versus Dragon variations

Figure 2 Graphical depiction of simulated repetitions of the experiments
13

The data do not show that the probability vector that generates game outcomes after 5a6 is different
from the probability vector that generates game outcomes if 5g6 is played instead. We must therefore
accept the possibility that the two observed experimental results were generated by the same underlying
probability vector [p
w
,p
b
,p
d
]. The observed distance between them of d=3.606 games may be explained
simply as sampling variation
14
. This finding, that the Sicilian Dragon is just as good a defense as the
Sicilian Najdorf, is at odds with opening book statistics
15
which show that the Najdorf variation is
preferred to the Dragon variation by a ratio of more than six to one (Jones & Powell, 2014). The decision
whether to play 5a6 or whether to play 5g6 in this situation cannot be based on the probability of
winning or of gaining relative advantage in terms of . It must therefore be based on other factors such as
playing style, personal preferences, and the deployment of specific strategies.

There are many adherents of the Sicilian Dragon variation (Dearing, 2004) (Gufeld, 2012) while more
traditional styles and most opening books prefer the Najdorf variation (Palliser, 2014) (Aronian, 2013)
(Jones & Powell, 2014). Our findings do not support the view that the Dragon variation contains an
inherent advantage over the Najdorf variation in terms of the probability vector as is often claimed (Ward,

13
The graph makes use of color and marker style to distinguish between the two variations. The first variation is shown in blue
with diamond markers and the second is shown in red with square markers.
14
Differences we would expect to observe if we repeated the same experiment. For example you would not expect to see exactly
150 heads every time you toss 300 fair coins.
15
Elo>2500
0
5
10
15
0 10 20 30 40
W
o
n

b
y

B
l
a
c
k

Won by White
B90 SicilianNajdorf -vs- B70 Sicilian Dragon

2001) nor that it contains a weakness as has been suggested (Marin, 1995) (Chess.com Forum, 2010)
(TradingHalt, 2011).

4.2 Decision #2: Accelerated Dragon Variation 3. Bb5 versus 3. d4
As shown in Table 1, we assume that the first two moves are 1.e4c5 2.Nf3Nc6 and that black wishes to
play the Accelerated Dragon and expects white to play 3. d4 so that play can continue with 3.d4cxd4
4.Nxd4g6. This sequence allows black to play the g6 move earlier
16
than in the conventional Dragon
variation described in the previous section. In so doing, black takes the risk that white may choose to play
3. Bb5 instead of 3. d4 (Perelshteyn, 2013) in which case play would continue as 3.Bb5g6 4.Bxc6dxc6
5.d3Bg7 6.h3 according to the opening book mainline after 3g6 (Jones & Powell, 2014).

The pairwise comparison of these two variations is therefore important to black in making the decision to
play the Accelerated Dragon (Perelshteyn, 2013). The results of the engine experiments for these two
variations are shown in Table 4 and depicted graphically in Figure 3. All six hundred games played in the
two experiments are available in PGN format in the online data archive for this paper (Munshi,
Variations: PGN Files, 2014).

B34 Accelerated Dragon 3. d4 1.00 300 12 6 282
B31 Rossolimo Variation 3. Bb5 2.51 300 41 5 254
Table 4 Sample data for the two Accelerated Dragon variations

The Euclidean distance between the two variations may be visualized as the length of the line shown in
Figure 3. This length is computed as 29 games
17
and its standard deviation computed from the simulated
sampling distribution allows us to set up the hypothesis test for =0 shown in Table 5. The test shows that
the probability of observing a distance of 29 or greater in a sample of n=300 games taken from a
population in which =0 is less that our threshold of disbelief set to =0.001. We therefore reject H
0
and
conclude that >0 and that therefore the probability vectors that determines game outcomes in the two
variations being compared are different, that is
1
2
.

Figure 3 Graphical depiction of the observed distance between 3.d4 and d.Bb5

16
Hence the term "accelerated"
17
The distance as computed has units of games. Division by the sample size of 300 games converts it to a dimensionless estimate
of probability.
3. d4
3. Bb5
4
6
8
10 20 30 40 50
W
o
n

b
y

B
l
a
c
k

Won by White
Accelerated Dragon: B34 vs B31

H
0
=0
H
a
>0
Observed distance 29.017
Standard deviation 5.501
t-value 5.275
p-value 1.48E-07
Decision Reject Ho
Table 5 Hypothesis test for distance between 3. d4 and 3. Bb5

Figure 4 Simulated repetitions of the two experiments
The hypothesis test may be made visually by examining the plot of the simulated repetitions of the
experiments shown in Figure 4 which shows that the simulated experiments form two distinct clusters and
do not overlap. The details of the simulations are available in the online data archive for this paper
(Munshi, Variations: Simulation files, 2014).

To interpret the conclusion that >0 in terms of relative advantage we must look at the direction of the
distance vector to determine whether the direction favors white, whether it favors black, or whether it is in
a neutral direction. The direction, easily seen in Figure 3, is made somewhat clearer if we move the axes
to the 3. Bb5 marker as shown in Figure 5. We can now see in Figure 5 that relative to 3. Bb5, 3. d4 lies in
the second quadrant. This means that the 3. d4 move by white favors black. Therefore, black's
Accelerated Dragon strategy succeeds only if white cooperates by playing 3. d4 and fails if white plays 3.
Bb5 instead. This finding is contrary to the popular belief in the robustness of the Accelerated Dragon
(Lalic, 2014) and consistent with Grandmaster Eugene Perelshteyn's intuition that the Accelerated Dragon
Defense by black is risky because it creates a relative disadvantage for black if white plays 3. Bb5 instead
of 3. d4 (Perelshteyn, 2013) (Scimia, 2014).

Figure 5 Direction of 3. d4 as seen from 3. Bb5
0
5
10
15
20
0 20 40 60 80
W
o
n

b
y

B
l
a
c
k

Won by White
B31 Accel Dragon Bb5 -vs- B34 Accel Dragon d4
3. d4
3. Bb5
0
2
-40 -30 -20 -10 0
B
l
a
c
k

d
i
f
f
e
r
e
n
c
e

White difference
Accelerated Dragon: B34 vs B31

4.3 Decision #3: Hyper Accelerated Dragon Variation 3. c3 versus 3. d4
We assume that black has played the Sicilian Defense Hyper
18
Accelerated Dragon as 1.e4c5 2.Nf3g6 and
now expects white to play 3. d4 so that play can continue with 3.d4cxd4 4.Nxd4Nc6 5. Nc3Bg7
19
. The
rationale for making the g6 move even earlier than in the Accelerated Dragon and deferring the Nc6 move
is that the threat of 3.Bb5
20
by white is thereby removed (Perelshteyn, 2013) (Scimia, 2014). Yet, like the
Accelerated Dragon variation, the Hyper Accelerated Dragon also contains the assumption that white will
play 3. d4. We therefore look for an alternate line by white in the opening book (Jones & Powell, 2014)
and select 3. c3 instead of
21
3. d4. The 3. c3 line played in the experiment is shown in Table 1. The
important difference between the two variations is that after 4. d4 cxd4 white now retakes with 5 .cxd4
instead of 5. Nxd4. We can now compare these two alternate lines to determine whether the alleged
advantage of the Hyper Accelerated variation is robust or whether it depends on white to cooperate by
playing 3. d4. The experimental results are shown in Table 6 and the Euclidean distance between the two
variations is depicted graphically in Figure 6
22
.

Variation Decision Rarity
23
Games played Won by white Won by black Draw
Hyper Accelerated Dragon 3. d4 1.00 300 22 4 274
Hyper Accelerated Dragon 3. c3 3.90 300 53 1 249
Table 6 Sample data for two variations of the Hyper Accelerated Dragon

Figure 6 Graphical depiction of the Euclidean distance between the two variations
The Euclidean distance between the two variations is the length of the line in Figure 6 and it is
approximately 31 games long. A Monte Carlo simulation method
24
is used to estimate the standard
deviation of the sampling distribution of distances and thereby to set up a hypothesis test for =0 as
shown in Table 7. The simulation results are shown graphically in Figure 7
25
.

18 The word Hyper implies that black plays the g6 move even earlier than in the Accelerated Dragon.
19 See Table 1
20 Described in Section 4.2
21 White plays 3. c3 to prepare for 4. d4 as shown in Table 1.
22 The games are available in PGN format in the online data archive for this paper (Munshi, Variations: PGN Files, 2014).
23 Relative to 3. d4. Computed from the opening book set to Elo>2500 (Jones & Powell, 2014).
24
See (Munshi, A method for comparing chess openings, 2014)
25
The simulation details are available in the online data archive for this paper (Munshi, Variations: Simulation files, 2014).
3. d4
3. c3
0
2
4
6
10 20 30 40 50 60
W
o
n

b
y

B
l
a
c
k

Won by White
Hyper Accelerated Dragon Variations

H
0
=0
H
a
>0
t-value 5.263
p-value 2E-7
Decision Reject Ho
Table 7 Hypothesis test for the two variations of the Hyper Accelerated Dragon

Figure 7 Graphical depiction of the Monte Carlo Simulations for the two variations of the Hyper Accelerated Dragon
The hypothesis test shows that if =0, the probability of observing a sample distance of 31 or greater is
less than our threshold of disbelief set to =0.001. Therefore, we reject H
0
and conclude that >0. The
implication is that is
1
2
. The probability vectors that generate game outcomes for the 3. d4 and 3. c3
variations of the Hyper Accelerated Dragon are different. We must now look at the direction of the
distance vector to see whether this difference is in a neutral direction or whether it suggests a relative
advantage.

Figure 8 Hyper Accelerated Dragon: direction of the distance vector

Figure 8 shows that the direction of the distance vector to 3. d4 as seen from 3. c3 lies in the second
quadrant. The direction implies that relative to the 3. c3 variation, the 3. d4 line
26
favors black. This
means that the Hyper Accelerated Dragon defensive strategy by black succeeds only if white cooperates
by playing 3. d4 and fails if white plays 3. c3 instead. In other words, the Hyper Accelerated Dragon is
not a robust strategy by black. Its success depends on white's cooperation.

26
The words "variation" and "line" are used interchangeably in this paper.
0
5
10
15
0 20 40 60 80
W
o
n

b
y

B
l
a
c
k

Won by White
Hyper Accelerated Dragon 3. c3 vs 3. d4
3. d4
3. c3
0
2
4
-40 -30 -20 -10 0
B
l
a
c
k

d
i
f
f
e
r
e
n
c
e

White difference
Direction of 3. d4 from 3. c3

This finding is not consistent with the opening book (Jones & Powell, 2014) which shows that 3. d4 is
preferred to 3. c3 by a factor of almost four to one at the grandmaster level of play
27
. Furthermore, the
finding does not support the assessment by GM Eugene Perelshteyn that the risk that white may play 3.
Bb5 in the Accelerated Dragon can be neutralized by playing the Hyper Accelerated Dragon
(Perelshteyn, 2013). Our findings show that the Hyper Accelerated Dragon involves the same kind of risk.
Both of these strategies depend on the assumed 3. d4 move by white. Just as the Accelerated Dragon fails
if white plays 3. Bb5 instead of 3. d4, so the Hyper Accelerated Dragon also fails if white plays 3. c3
instead of 3. d4. Neither of these strategies is robust.

4.4 Decision #4: Sicilian Defense Yugoslav Attack 9. Bc4 versus 9. O-O-O
A popular response by white to the Sicilian Dragon is the so called Yugoslav Attack (Dearing, 2004)
which follows the mainline from the opening book for the first eight moves and then white makes the
move 9. Bc4 , the signature move of the Classical variation of the Yugoslav Attack. The alternate move
from the opening book that is also covered in Dearing's book (Dearing, 2004) is the so called Modern
variation of the Yugoslav Attack in which white castles queenside with 9. O-O-O instead of 9. Bc4.
We can compare these two variations to determine whether the probability vectors for these two
variations are different and if so whether the difference involves a relative advantage for one side over the
other. The move sequences for the two variations shown in Table 1 are taken from the opening book with
the level of play set to Elo greater than 2600 (Jones & Powell, 2014).

The sample data
28
are shown in Table 8 and depicted graphically in Figure 9. The rarity data show that the
two variations are played with equal frequency at a high level of play
29
. The length of the line in Figure 9
is the distance between these variations and it computed to be 37 games. Using the standard deviation of
the simulated sampling distribution of distances
30
we can compute the probability that we would observe
a sample distance of 37 or greater if =0 and that probability turns out to be close to zero. In the
hypothesis test shown in Table 9 we reject H
0
and conclude that >0 and therefore that
1
2
, that is, the
probability vectors that determine game outcomes in the two variations are different.

Yugoslav Attack/Modern 9. O-O-O 1.00 300 50 10 226
Yugoslav Attack/Classical 9. Bc4 1.00 300 13 4 283
Table 8 Results of the Yugoslav Attack experiments

Figure 9 Graphical depiction of the Euclidean distance between the two variations

27
Indicated by an Elo rating of greater than 2500.
28 The games are available in PGN format in the online data archive for this paper (Munshi, Variations: PGN Files, 2014).
29 Elo>2500
30 The computational details are available online in the data archive for this paper (Munshi, Variations: Simulation files, 2014).
9. O-O-O
9. Bc4
0
4
8
12
10 20 30 40 50 60
W
o
n

b
y

B
l
a
c
k

Won by White
Yugoslav Attack

H
0
d=0
H
a
d>0
t-value 6.445
p-value 1E-10
Decision Reject Ho
Table 9 Comparison of the Yugoslav Attack variations
Now that we know that that
1
2
, we must look at the direction of the distance vector to determine
whether the implied difference in involves a relative advantage or whether it lies in a neutral direction.
The direction of the castle option as seen from the 9. Bc4 option is made explicit in Figure 10 where it is
clear to see that the 9. O-O-O variation lies in the first quadrant. The first quadrant represents a neutral
direction and therefore the Modern Yugoslav variation offers no relative advantage to either side relative
to the Classical Yugoslav variation.

Figure 10 The direction of the distance vector from 9. Bc4 to 9. O-O-O
The hypothesis test and the direction of the distance vector may be visualized in a graphical depiction of
the simulated repetitions of the experiments shown in Figure 11.

Figure 11 Simulated repetitions of the two experiments
The probability vectors of these two variations are different but the difference lies in a neutral direction
and does not offer a relative advantage to either side. Therefore the choice of variation in this case cannot
be based on the probability of winning. Instead, the decision must be based on the probability of decisive
games against the probability of a draw.
9. O-O-O
9. Bc4
0
2
4
6
8
0 10 20 30 40
B
l
a
c
k

d
i
f
f
e
r
e
n
c
e

White difference
Direction of 9. O-O-O from 9. Bc4
0
5
10
15
20
25
0 20 40 60 80
W
o
n

b
y

B
l
a
c
k

Won by White
B76 Yugoslav Attack -vs- B77 Yugoslav Attack

The classical version offers a closed or "drawish" game with a greater likelihood that the game will end in
draw. The modern version offers an open game with a higher probability of a decisive game. This finding
is consistent with the equal preference for these variations in the opening book; and with the observation
by analysts that when white and black castle on opposite sides of the board it creates attacking
opportunities for both sides and that therefore these games are more likely to be decisive (Chernev, 2003)
(Vukovic, 1999) (Dearing, 2004).

5. CONCLUSIONS

It is demonstrated that a methodology previously proposed for evaluating chess openings by comparing
them with neutral and perfect openings may also be used to make pairwise comparisons of alternate lines
of any opening.

Pairwise comparisons show that there is no evidence of a difference between the B70 Sicilian
Defense/Dragon Variation and the B90 Sicilian Defense/Najdorf Variation. The data are consistent with
our null hypothesis that the same underlying probability vector drives game outcomes in both of these
variations.

The Sicilian Defense/Accelerated Dragon and Hyper Accelerated Dragon variations are found to be
strategies by black which depend on white's cooperation. In both cases white is able to change the
probability vector sharply in its favor by making a move that is different from the one assumed by black.
These variations are therefore not robust because in both cases black takes the risk that white will not
cooperate.

An interesting case is that of the two variations of the Sicilian Defense/Yugoslav Attack. They are found
to be driven by different probability vectors but the difference lies in a neutral direction. The classical
variation is "drawish" by comparison because it offers a higher probability of draw than the more open
game initiated by the modern variation of the Yugoslav Attack.

These results are likely to be relevant in games in which the move imperfection rate IMP is held very low
so that the effect of the opening can exert a measurable impact on the probability vector. Our analysis in
the Appendix shows that the IMP rate in our experiment is approximately 2% and that the IMP rate even
for titled players is significantly higher. It is not likely that our findings are directly relevant in games
with such high values of IMP although they may be useful as part of the information set used by chess
players to form their opening strategy.

The large number of books on the relative merit of chess openings and the significant amount of Internet
resources devoted to this topic form a convincing body of evidence that the comparison of chess openings
is important and of great interest to chess players at all levels of play. It is in this spirit that this work is
undertaken and this paper presented. The value of this study lies not so much in the specific findings
themselves but in demonstrating and validating an objective methodology for comparing chess openings
in general.


6. APPENDIX

6.1 An estimate of white's first move advantage (FMA)
If we combine our experimental data for the B53, B90, and B70 Sicilian Defense games and assume that
these openings are neutral and perfect that do not provide a relative advantage to either side by virtue of
the opening employed and that therefore OE=0, then we can use the combined sample of 900 games to
estimate the level of IMP in these experiments as well as the value of white's first move advantage(FMA).

Opening Games White wins Black wins
B53 Sicilian Defense
31
300 19 3
B90 Sicilian/Najdorf 300 20 4
B70 Sicilian/Dragon 300 23 2
Total 900 62 9
Percent 6.89% 1.00%
Table 10 Aggregated data for neutral openings
The data aggregation in Table 10 shows that white won 6.89% of these games and black won 1%. If on
average black wins half of the games made decisive by IMP then IMP/2=1% and therefore the level of
move imperfection in our experiments is estimated as IMP = 2%. Since white also won 1% of the games
due to IMP, the remaining wins by white may be ascribed to the first move advantage as FMA = 6.89% -
1% = 5.89%. The standard deviation of IMP and FMA may be estimated by the usual Gaussian
approximation. These values and the corresponding 95% confidence intervals are shown in Table 11. The
first move advantage FMA is a universal constant. The value of IMP applies to the experimental
conditions employed in these experiments.

95%CI 95%CI
Variable Mean Stdev Low High
FMA 5.89% 0.007847 4.35% 7.43%
IMP 2.00% 0.004667 1.09% 2.91%
Table 11 The 95% Confidence Intervals for FMA and IMP

6.2 An estimate of the move imperfection rate (IMP) for titled chess players
The data in Table 12 are taken from the opening book database at chesstempo.com (Jones & Powell,
2014). They show the number of games in the database and the win percentage by black and by white
after the move 5a6 in the B90 Sicilian Defense/Najdorf Variation at different minimum levels of play
32
.
We assume that the Sicilian Najdorf is a perfect and neutral opening and that therefore there is no opening
effect in these games (DeFirmian, 2008). This means that the probability vector that generates game
outcomes is defined completely by FMA, IMP, and DIFF.

31
See (Munshi, A method for comparing chess openings, 2014).
32
We have assumed a maximum rating of 2800. The source data does not contain this information.

Assuming a maximum rating of 2800 in the database, we may deconstruct the aggregation of the data in
Table 12 to compute the number of games and the percentage win by black and by white in each of the
six rating bands implied by the data. The computed values are shown in Table 13.

Elo Range Number Percentage win
Low High Games White Black
2200 2800 44062 35.6% 30.3%
2300 2800 31345 34.2% 28.5%
2400 2800 19053 32.9% 26.2%
2500 2800 7676 31.2% 23.3%
2600 2800 2264 31.6% 22.3%
2700 2800 640 30.5% 22.2%
Table 12 Opening Book database statistics provided by Jones and Powell

Elo Range Number Percentage win Estimation of IMP from the data
Low High Games White Black Decisive DIFF FMA IMP
2200 2300 12717 39.1% 34.7% 73.8% 28.00% 5.89% 39.9%
2300 2400 12292 36.2% 32.1% 68.3% 28.00% 5.89% 34.4%
2400 2500 11377 34.0% 28.2% 62.2% 28.00% 5.89% 28.3%
2500 2600 5412 31.0% 23.7% 54.8% 28.00% 5.89% 20.9%
2600 2700 1624 32.0% 22.3% 54.4% 28.00% 5.89% 20.5%
2700 2800 640 30.5% 22.2% 52.7% 28.00% 5.89% 18.8%
Table 13 Win statistics for 100 Elo point rating bands
The column marked "Decisive" in Table 13 is the sum of white win and black win percentages. It
represents the percent of games that were decisive. The column marked DIFF is an estimate of the
maximum percent of games that could have been made decisive by virtue of a difference in playing
strength.

In each rating band the maximum difference in Elo rating is 100 points and if all games played in each
band are played at an Elo differential of 100 rating points, then the stronger player will win 64% of the
decisive games and the weaker player will win 36% (Caissa's Web, 2014). The difference of 28%
represents the stronger player's relative advantage by virtue of playing strength. It represents an extreme
condition expected to yield a minimum value of IMP that can be derived from these data.

White's first move advantage was estimated to be 5.89% in the previous section
33
. Since there is no
opening effect, the occurrence of decisive games is explained entirely in terms of DIFF, FMA, and IMP
and so we can estimate IMP by subtraction as shown in Table 13. The estimated move imperfection rates
are very high, about 20% in grandmaster games and almost twice that in the lowest titled category.

The IMP value for each Elo band listed in Table 13 is the probability that a game that would otherwise
have ended in draw will become decisive because of move imperfections. Move imperfections are moves
that change the probability vector in favor of the opponent. They may involve threats or opportunities that
were overlooked (Avni, 2011).

33
See Table 11

It is possible that except for extreme cases
34
, nominally sub-optimal and non-robust opening strategies
survive in chess games even at high levels of play because the opening effect is ineffectual at a high rate
of IMP. Because of the greater importance of IMP in most chess games, factors other than the effect of
the opening on the probability vector become relatively more important in the selection of an opening line
to play. For example the player may have studied the Accelerated Dragon in depth and she may know that
the opponent is not familiar with it or that the opponent habitually plays 3. d4 in that line. In either of
these cases, the player gains an advantage by playing the Accelerated Dragon and taking the risk of 3.
Bb5 because she knows that the risk is low and that the upside potential is high. Chess is often a form of
psychological warfare that involves calculated risk taking of this kind (Davies, 2004).

6.3 Extreme Cases
Four openings that are extremely rare in grandmaster games are tested
35
. The sample data are shown in
Table 14. The comparison of these innovations with the B53 Sicilian Defense
36
is summarized in Table 15
and depicted graphically in Figure 12. The column labeled "Probability" in Table 15 is an estimate of the
probability of winning ceded to the opponent by virtue of the innovations shown in Table 14.

In all four cases, we reject the null hypothesis and conclude that >0 and in all four cases we find that the
direction of the distance vector indicates a change in relative advantage in favor of the opponent. The
directions and standardized distances
37
are shown in Figure 12. We surmise that their virtual absence from
the grandmasters' opening repertoire can be explained in terms of these findings and that therefore the OE
of these extreme openings is relevant even at the higher values of IMP noted in Table 13.

Rare Opening Move equence Innovator Innovation Games White Black
C40 Latvian e4e5 Nf3f5 Nxe5Qf6 Black 2f5 300 148 0
A00 Grob g4d5 Bg2c6 h3e5 White 1. g4 300 3 111
A03 Bird f4d5 Nf3Bf5 e3Nf6 White 1. f4 300 3 45
A00 Sokolsky b4e5 Bb2Bxb4 Bxe5Nf6 White 1. b4 300 4 35
Table 14 Rare Openings
Test opening Distance Probability
C40 Latvian 129.03 43%
A00 Grob 109.18 36%
A03 Bird 44.94 15%
A00 Sokolsky 35.34 12%
Table 15 Comparison with B53 Sicilian

34
Described in the next section
35
The PGN data for all four experiments are available in an online data archive (Munshi, Extreme PGN files, 2014).
36
The PGN file for the B53 Sicilian Defense experiment is available online (Munshi, PGN files, 2014).
37
Standardized distance = distance/standard deviation

Figure 12 Magnitude and direction of standardized distances

7. REFERENCES

Aronian, L. (2013). Winning with the Najdorf Sicilian. New in Chess.
Avni, A. (2011). Danger in chess. Dover Publications.
Caissa's Web. (2014). chess ratings. Retrieved 2014, from caissa.com:
http://www.caissa.com/support/chess-ratings.php
Chernev, I. (2003). Logical chess move by move. Batsford Chess.
Chess.com Forum. (2010). Sicilian Dragon Refuted. Retrieved 2014, from chess.com:
http://www.chess.com/forum/view/chess-openings/sicilian-dragon-refuted
Davies, N. (2004). Chess for scoundrels (CD).
Dearing, E. (2004). Play the Sicilian Dragon. Gambit Publications.
DeFirmian, N. (2008). Modern chess openings. Random House.
Edwards, J. (2012). Chess is fun. Kindle.
Georgiev, K. (2012). Sharpest Sicilian. Chess Stars.
Gufeld, E. a. (2012). The Sicilian Dragon New and Revised. Kindle.
Houdart, R. (2012). Houdini. Retrieved November 2013, from cruxis.com:
http://www.cruxis.com/chess/houdini.htm
Latvian
Grob
Bird
Sokolsky
-8
0
8
16
-5 0 5 10 15 20
B
l
a
c
k

d
i
f
f
e
r
e
n
c
e

White differnce
Standardized distance from B53 Sicilian

Houdart, R. (2013). Houdini Chess. Retrieved 2014, from cruxis.com:
http://www.cruxis.com/chess/houdini.htm
Johnson, V. E. (2013, November). Revised Standards for Statistical Evidence. Retrieved December 2013,
from Proceedings of the National Academy of Sciences:
http://www.pnas.org/content/110/48/19313.full
Jones, R., & Powell, D. (2014). Game Database. Retrieved February 2014, from chesstempo.com:
http://chesstempo.com/game-database.html
Lalic, P. (2014). Play the Accelerated Dragon. Everyman Chess.
Marin, M. (1995). Yugoslav Attack. New in Chess.
Meyer-Kahlen, S. (2007). Deep Shredder. Retrieved 2014, from Shredderchess.com:
http://www.shredderchess.com/chess-software/deep-shredder12.html
Meyer-Kahlen, S. (2000). Opening Database. Retrieved January 2014, from Shredder Chess:
http://www.shredderchess.com/online-chess/online-databases/opening-database.html
Munshi, J. (2014). A method for comparing chess openings. Retrieved 2014, from Arxiv:
http://arxiv.org/ftp/arxiv/papers/1402/1402.6791.pdf
Munshi, J. (2014). A Method for Comparing Chess Openings. Retrieved 2014, from SSRN:
http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2415203
Munshi, J. (2014). Comparing Chess Openings. Retrieved 2014, from SSRN:
Munshi, J. (2014). Comparing Chess Openings Part 3. Retrieved 2014, from ssrn:
Munshi, J. (2014). Extreme PGN files. Retrieved 2014, from Dropbox:
https://www.dropbox.com/sh/22mvfdk4qa8b1p9/AADuRHKPbEpsJKJaz8Swyrz8a
Munshi, J. (2014). PGN files. Retrieved 2014, from Dropbox:
https://www.dropbox.com/sh/7zvo7hoq5gych4e/AAC8EXiSY4UkYEIkQ6vh9hlSa
Munshi, J. (2014). Variations: PGN Files. Retrieved 2014, from Dropbox:
https://www.dropbox.com/sh/5rkzlvc5j1by8fx/AADk8oIO92_8ixKppHSQaxfDa
Munshi, J. (2014). Variations: Simulation files. Retrieved 2014, from Dropbox:
https://www.dropbox.com/sh/e47clpjvukaqazf/AABsdo4D7ar7gaHJiUDgrwIYa
Palliser, R. (2014). Attacking Chess: The Sicilian Najdorf. Everyman Chess.

Perelshteyn, E. (2013). Accelerated Dragon. Retrieved 2014, from Youtube:
http://www.youtube.com/watch?v=h11MkaA32Vo
Scimia, E. (2014). Common lines in the Sicilian defense. Retrieved 2014, from About.com:
http://chess.about.com/od/openings/ss/Sicilian_8.htm
Taulbut, S. (1987). How to play the Ruy Lopez. Collier Books.
TradingHalt. (2011). Beating the Dragon. Retrieved 2014, from Youtube:
http://www.youtube.com/watch?v=3v8TFl5AvGc
Vukovic, V. (1999). The art of attack inchess. Everyman Chess.
Ward, C. (2001). Winning with the SicilianDragon. Batsford Chess Books.
Wikipedia. (2014). Houdini Chess. Retrieved 2014, from Wikipedia:
http://en.wikipedia.org/wiki/Houdini_(chess)
Wikipedia. (2014). Monte Carlo Simulation. Retrieved 2014, from Wikipedia:
http://en.wikipedia.org/wiki/Monte_Carlo_method
Wikipedia. (2014). Pairwise comparisons. Retrieved 2014, from Wikipedia:
http://en.wikipedia.org/wiki/Pairwise_comparison

SSRN Id2472783 PDF

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

SSRN Id2472783 PDF

Uploaded by

Copyright:

Available Formats

Electronic copy available at: http://ssrn.

You might also like