Lecture 2, part b, Interpretation of spreadsheet

A Spreadsheet Demonstrating the Interaction of the Magnitude of the Error Term

and the Treatment Effect in an One-Way ANOVA Using Simulated Data
Steven W. Lamb, Indiana State University
Constance H. McLaren, Indiana State University
The purpose of the Excel spreadsheet is to allow the student to quickly grasp the impact that
various magnitudes of treatment effects and various magnitude of the standard deviation of
the error term will have in determining significance in a One-way ANOVA. The students will
gain insight in the nature and attributes of the theoretical model.
The spreadsheet developed by the authors allows the students to input three values into the
theoretical model

X ij =+ j

e ij (where e ij are N(0, ) Those input values are the

value of the Grand Mean, ; the treatment effects

equals 0, and

j s, (where

1 equals -k,

3 equals +k); there are three treatments, and finally the population value

of the standard deviation of the error term

e .

Students will also easily be able to determine the impact of error terms which do not sum to
zero (where the statistics associated with the error terms differ significantly from their
parameters) upon the estimates of the Treatment Effects, as well as upon the estimate of
other parameters.
The spreadsheet is set up so that three sets of 30 normal error terms are sampled from a
Normal Distribution with mean zero, and standard deviation equal to one. Of course, these
sampled error terms most likely will not have a sample mean equal to one, and the sample
standard deviation will certainly deviate from the designated parameter as well. The student
will be able to easily grasp the impact of this deviation upon the estimate of the parameters
as well as upon the outcome associated with the one-way ANOVA.
Learning Points Immediately Apparent as Well as Discussion Points

Other things being equal, as the error term increases, as within treatment variance
increases, the F value decreases, the p value increases; significance becomes less
probable.
a

The students will quickly discern that large internal variance may remove the
ability to detect treatment effects. The smaller the treatment effects, the
more quickly an increase in internal variance will mask the treatment effects.

It has to be stressed to the students that once the experiment is designed,

one does not have control over the magnitude of the standard error term.
However, in the design of the One-Way ANOVA experiment one may impact
the magnitude of the error term by making the subjects assigned to the
various treatments more or less homogeneous.

The more homogenous the subjects assigned to the various treatments, the
smaller the standard deviation of the error term, the more probable
significance will be found for treatment effects. (This also allows the instructor

to introduce the concept of blocking as a means of controlling the magnitude

of the error term).
d

However, the greater the homogeneity of the subjects assigned at random to

the various treatment groups, the greater is the difficulty of generalizing the
results to broader populations.

Other things being equal, as the treatment effects increase, F values increase, p
values increase; significance becomes more probable.
a

Again, it must be stressed to students that once an experiment is designed,

one does not have control over the magnitude of the treatment effects. In the
design of the treatments, however, one may have control over the treatment
effect.

The greater the treatment effect, i.e., the more meaningful the treatment
effect, the greater is the probability of identifying significant treatment effects.

An area of the spreadsheet is set up to display the values of the parameters and the
value of the statistics estimating these parameters.
a

In the initial table is the value of the variance, the grand mean and the
treatment means.

Immediately following that table is the embedded explanation behind the

deviation between the statistics and their parameters. The average value of
the overall error term is given (

e ij ). This value is the value of the

difference between the grand mean and its estimate. Next, the average value
of the error terms associated with each treatment is given (

e i1 ; e i2 ; and

e i3 ). These differences are the differences between each treatment mean

and its estimate.
c

Students will view how the value of the average error term will impact the
estimate of the parameter. For example, if an over abundance of bright
students happens to be assigned to a specific teaching technique; i.e., if the
average value of the error term assigned to that teaching treatment turns out
to be quite positive, then the estimate of that specific treatment effect will
turn out to be greater than the parameter.

Students may confuse the values of the means of the error terms with the
means of their estimates (

e i1

versus

e^ i1

for example). The mean of the

estimates of error terms will always equal zero. This is an attribute of the
ANOVA process. This confusion will need to be addressed.
e

The instructor will need to emphasize that the greater the sample size, the
closer the value of the average error term will tend to be to its parameter
(0),i.e., the smaller the deviation of

and the less the estimates of the

treatment effects will tend to be unduly effected by the error terms.

f

The instructor will need to discuss the impact of small sample sizes upon the
deviation of the average error term as well.