You are on page 1of 35

Practical Guide to Estimating One-Way ANOVA

(source: Lund Research, www.laerd.com)

Introduction

If you want to determine whether there are any statistically significant differences between the
means of two or more independent groups, you can use a one-way analysis of variance (ANOVA).
The one-way ANOVA can be an appropriate test for analysing the results from many different
types of study design. For example, you could use a one-way ANOVA to determine whether exam
performance differed based on test anxiety levels amongst students (i.e., your dependent variable
would be "exam performance", measured from 0-100, and your independent variable would be
"test anxiety level", which has three groups: "low-stressed students", "moderately-stressed
students" and "highly-stressed students"). As another example, a one-way ANOVA could be used
to understand whether there is a difference in salary based on degree subject (i.e., your dependent
variable would be "salary" and your independent variable would be "degree subject", which has
five groups: "business studies", "psychology", "biological sciences", "engineering" and "law").

It is important to realize that the one-way ANOVA is an omnibus test statistic and cannot tell you
which specific groups were significantly different from each other; it only tells you that at least
two groups were different. Since you may have three, four, five or more groups in your study
design, determining which of these groups differ from each other is important. You can do this
using follow-up tests.

Basic requirements of the one-way ANOVA

In order to run a one-way ANOVA, there are six assumptions that need to be considered. The first
three assumptions relate to your choice of study design and the measurements you chose to make,
whilst the second three assumptions relate to how your data fits the one-way ANOVA model.
These assumptions are:

o Assumption #1: You have one dependent variable that is measured at


the continuous level. Examples of continuous variables include include height
(measured in metres and centimetres), temperature (measured in C), salary (measured in
US dollars), revision time (measured in hours), intelligence (measured using IQ score),
firm size (measured in terms of the number of employees), age (measured in years),
reaction time (measured in milliseconds), grip strength (measured in kg), power output
(measured in watts), test performance (measured from 0 to 100), sales (measured in number
of transactions per month), academic achievement (measured in terms of GMAT score),
and so forth.

Assumption #2: You have one independent variable that consists of two or more
categorical, independent groups. Typically, a one-way ANOVA is used when you have three or
more categorical, independent groups, but it can be used for just two groups (although
an independent-samples t-test is more commonly used for two groups). Example independent
variables that meet this criterion include ethnicity (e.g., three groups: Caucasian, African American
and Hispanic), physical activity level (e.g., four groups: sedentary, low, moderate and high),
profession (e.g., five groups: surgeon, doctor, nurse, dentist, therapist), and so forth.

Assumption #3: You should have independence of observations, which means that there is no
relationship between the observations in each group of the independent variable or between the
groups themselves. Indeed, an important distinction is made in statistics when comparing values
from either different individuals or from the same individuals. Independent groups (in a one-way
ANOVA) are groups where there is no relationship between the participants in any of the groups.
Most often, this occurs simply by having different participants in each group.

For example, if you split a group of individuals into four groups based on their physical activity
level (e.g., a "sedentary" group, "low" group, "moderate" group and "high" group), no one in the
sedentary group can also be in the high group, no one in the moderate group can also be in the high
group, and so forth. As another example, you might randomly assign participants to either a control
trial or one of two interventions. Again, no participant can be in more than one group (e.g., a
participant in the the control group cannot be in either of the intervention groups). This will be true
of any independent groups you form (i.e., a participant cannot be a member of more than one
group). In actual fact, the 'no relationship' part extends a little further and requires that participants
in different groups are considered unrelated, not just different people (e.g., participants might be
considered related if they are husband and wife, or twins). Furthermore, participants in one group
cannot influence any of the participants in any other group.

o Assumption #4 There should be no significant outliers in the groups of your independent


variable in terms of the dependent variable

If there are any scores that are unusual in any group of the independent variable, in that
their value is extremely small or large compared to the other scores, these scores are called
outliers (e.g., 8 participants in a group scored between 60-75 out of 100 in a difficult maths
test, but one participant scored 98 out of 100). Outliers can have a large negative effect on
your results because they can exert a large influence (i.e., change) on the mean and standard
deviation for that group, which can affect the statistical test results. Outliers are more
important to consider when you have smaller sample sizes, as the effect of the outlier will
be greater. Therefore, in this example, you need to investigate whether the dependent
variable, coping_stress , has any outliers for each group of the independent
variable, group (i.e., you are testing whether the coping stress score is outlier free for the
"sedentary", "low", "moderate" and "high" groups).

Due to the effect that outliers can have on your results, you have to choose whether to
include these in your data when performing a one-way ANOVA in SPSS Statistics. First,
we show you how you can detect whether you have outliers using the Explore... procedure
in SPSS Statistics, which will create boxplots of your data. Then, we show you how to
determine whether you have outliers based on these boxplots, and if you do, different ways
you can deal with these. After determining whether you have outliers, and if you do, how
to deal with them, you can move on to determine whether each group is normally
distributed (see Assumption #5 below).

o Assumption #5 Your dependent variable should be approximately normally distributed for


each group of the independent variable

The assumption of normality is necessary for statistical significance testing using a one-
way ANOVA. However, the one-way ANOVA is considered "robust" to violations of
normality. This means that some violation of this assumption can be tolerated and the test
will still provide valid results. Therefore, you will often hear of this test only
requiring approximately normal data. There are many different methods available to test
this assumption. We show you one of the most common methods: the Shapiro-Wilk test
for normality. This is a numerical method for testing normality and is run using
the Explore... procedure in SPSS Statistics.

o Assumption #6 You have homogeneity of variances (i.e., the variance is equal in each group
of your independent variable)

The assumption of homogeneity of variances states that the population variance for each group
of your independent variable is the same. If the sample size in each group is similar, violation
of this assumption is not often too serious. However, if sample sizes are quite different, the
one-way ANOVA is sensitive to the violation of this assumption. Either way, SPSS Statistics
uses Levene's test of equality of variances and two differently-calculated one-way
ANOVAs, which will give you a valid result irrespective of whether you met or violated this
assumption; that is, SPSS Statistics provides a one-way ANOVA that is calculated normally
(with pooled variances) and another for when the assumption is violated that uses separate
variances (i.e., non-pooled variances) and the Welch-Satterthwaite correction to the degrees of
freedom. In addition, if your data fails this assumption, we show you how to run a Games-
Howell post hoc test for your multiple comparisons instead of the Tukey post hoc test.

Null and alternative hypotheses

It is important to remember that when you run a one-way ANOVA, you are trying to determine
whether the group means are different in the population. Consider the graph below, which is a plot
of the data from the example you will be using in this guide:
In this example, participants were split into four separate groups based on their level of physical
activity (i.e., the "Sedentary" to "High" groups along the horizontal axis of the graph) and they had
a dependent variable called CWWS score recorded (i.e., the scores along the vertical axis of the
graph). More will be explained about this example later, but for the moment, we are simply
investigating if the mean CWWS score differs between the different groups.

We can see from the graph that the group means are different between the four groups, with the
mean CWWS score increasing with increasing levels of physical activity group (i.e., from left to
right on the graph). We can also see that participants' scores differed in each group, as illustrated
by the whisker bars (e.g., the CWWS scores for participants in the "low" physical activity group
were much more spread out than the CWWS scores for participants in the "sedentary" physical
activity group, where the length of the whisker bar is much shorter). This description of the data
reflects what is occurring in this particular sample of participants in this particular study. That is,
we are describing sample data. However, we do not know if the differences between these four
groups is: (a) due to sampling variation or (b) reflects actual group differences in the
population from which the participants were sampled (i.e., the population reflects the people that
this sample is meant to represent). As such, we use a one-way ANOVA to determine whether the
mean CWWS score in the physical activity groups is different in the population. To achieve this,
you assume the null hypothesis, which states that there are, in fact, no differences
inpopulation means between the groups. Another way of stating this is that the
group population means are equal. This is described more formally as follows:

H0: all group population means are equal (i.e., 1 = 2 = 3 = ... = k)

where = population mean and k = number of groups. Therefore, for our physical activity
example, we could write:

H0: all physical activity group population means are equal (i.e., sedentary = low = moderate = high)

However, using our sample data, we are really trying to find evidence against this null hypothesis
and accept the alternative hypothesis, which states that there are differences between the group
population means (that is, you are usually conducting an experiment because you hope to find
differences). Another way of stating this is that not all the group population means are equal. This
is described more formally as follows:

HA: at least one group population mean is different (i.e., they are not all equal)
At this point, it is important to note that you are not prescribing a particular way in which the
groups means differ in the population, only that they do differ in some way. The one-way ANOVA
calculates an F ratio based on the variability between groups versus the variability within groups
(see any good textbook for additional details, e.g., Kirk, 2013). The probability (p-value) of finding
an F ratio as large as the one calculated by the one-way ANOVA is used to either reject or not
reject the null hypothesis. If this probability value is less than .05 (i.e., p < .05), there is a less than
5 in 100 (5%) chance of the F ratio being as large as calculated, given that the null hypothesis is
true. This is interpreted as there being "real" differences between group means in the population
(i.e., you wouldn't expect the sample means to vary as much as they do if they were equal in the
population). When this occurs, the result is called statistically significant.

Determining whether there are differences between groups: Post hoc tests

A statistically significant one-way ANOVA determines whether three or more group means differ
in some undisclosed way in the population. That is, it doesn't go into specifics about where the
differences between groups lie. The one-way ANOVA is sometimes referred to as an omnibus
test for this reason. However, rarely do you not want to know where the differences between
groups lie. For example, you might want to know whether there is a difference between the "Low"
and "High" physical activity groups and whether this difference is statistically significant.

The most common type of post hoc test is one that makes all pairwise comparisons. A pairwise
comparison is a comparison between two separate groups (e.g., a comparison between the "Low"
and "High" physical activity groups). Common post hoc tests test for all possible combinations of
these pairwise comparisons (e.g., six pairwise comparisons, consisting of a comparison between
the "Sedentary" vs. "Low" groups, the "Sedentary" vs. "Moderate" groups, the "Sedentary" vs.
"High" groups, the "Low" vs. "Moderate" groups, the "Low" vs. High" groups, and the "Moderate"
vs. "High" groups).

What problems can you solve using a one-way ANOVA?

A one-way ANOVA is most often used for three types of study design:

1. Determine if there are differences between three or more independent groups

You have a study design where you are measuring the same dependent variable in three or more
independent (i.e., unrelated) groups (and you want to know if there are differences between
groups), a one-way ANOVA might be appropriate. For example, you could use a one-way
ANOVA to compare maximal aerobic capacity in swimmers, runners and cyclists. Another
example could be to investigate any differences in blood cholesterol concentration in sedentary,
low, moderate and high physical activity groups.

2. Determine if there are differences between conditions (with no pre-test measurement taken)

If you have a study design where three or more independent groups have performed different
interventions (e.g., control/interventions) and the same dependent variable is measured at the end
of the study in all groups, a one-way ANOVA might be appropriate. For example, one group acted
as a control, one group underwent an exercise-training programme and another group underwent
a dietary programme. Body weight was measured at the end of each programme and compared
between the three separate groups to determine if there were any statistically significant mean
differences between the groups.

3. Determine if there are differences in change scores

If you have a study design where three or more independent groups have performed different
interventions (e.g., control/interventions), the same dependent variable is measured at the
beginning and end of the study in all groups, and a change score calculated (i.e., post-values minus
pre-values), a one-way ANOVA might be appropriate. For example, pre- and post- blood glucose
concentration measurements were taken and change scores calculated for an exercise intervention
group, dietary intervention and a control group. These change scores were then compared between
the three groups using a one-way ANOVA. This will determine whether the changes in blood
glucose concentration between groups was equal or if there were statistically significant
differences in change score (i.e., the intervention type had a differential effect on change in blood
glucose concentration).

Example used in this guide

A researcher believes that individuals that are more physically active are better able to cope with
stress in the workplace. To test this theory, the researcher recruited 31 subjects and measured how
many minutes of physical activity they performed per week and their ability to cope with
workplace stress. The subjects were categorized into four groups based on the number of minutes
of physical activity they performed: namely, "sedentary", "low", "moderate" and "high" physical
activity groups. These groups (levels of physical activity) formed an independent variable
called group . The ability to cope with workplace stress was assessed as the average score of a
series of Likert items on a questionnaire, which allowed an overall "coping with workplace stress"
score to be calculated; higher scores indicating a greater ability to cope with workplace-related
stress. This dependent variable was called coping_stress and "ability to cope with workplace-
related stress" abbreviated as "CWWS" score. The researcher would like to know if CWWS score
is dependent on physical activity level. In variable terms, is mean coping_stress score different
for different levels of group ?

Setting up your data

For a one-way ANOVA, you will have two variables. In this example, these are:

1) The dependent variable, coping_stress , which is the "ability to cope with workplace-
related stress" (abbreviated as "CWWS" score);
2) The independent variable, group , which has four ordered categories: "Sedentary",
"Low", "Moderate", and "High" (N.B., the categories do not have to be ordered in a one-
way ANOVA).

To set up these variables, SPSS Statistics has a Variable View where you define the types of
variables you are analysing and a Data Viewwhere you enter your data for these variables. First,
we show you how to setup your independent variable and then your dependent variable in
the Variable View window of SPSS Statistics. Finally, we show you how to enter your data into
the Data View window.

The Variable View in SPSS Statistics

At the end of the setup process, your Variable View window will look like the one below, which
illustrates the setup for both the independent and dependent variable:

In the Variable View window above, you will have entered two variables: one on each row. For
our example, we have put the independent variable, group , on row , and the
continuous dependent variable, coping_stress , on row . First, look at the independent
variable, group , on row below:
Enter the name of your independent variable in the cell under the column (e.g.,
"group" in our example). There are certain "illegal" characters that cannot be entered into
the cell. Therefore, if you get an error message, you can learn what these are in our
general data setup guide here. Next, the cell under the column should contain the
information about the four groups of your independent variable (e.g., "Sedentary", "Low",
"Moderate" and "High" in our example). This will have been entered using the Value
Labels dialogue box, as shown below:

The cell under the column should show or , indicating


that you have a categorical independent variable. In our example, the cell under
the column shows because group is an ordinal variable. However,
if your independent variable is a nominal variable (e.g., gender ), the cell under
the column should show . Finally, the cell under
the column should show or .

Next, look at the continuous dependent variable, coping_stress , on row below:


Enter the name of your dependent variable in the cell under the column (e.g.,
"coping_stress" in our example). The cell under the column should
show , indicating that you have a continuous dependent variable, whilst the cell under
the column should show or .

You have now successfully entered all the information that SPSS Statistics needs to know about
your independent and dependent variables into the Variable View window. In the next section,
we show you how to enter your data into the Data View window.

The Data View in SPSS

Based on the file setup for the dependent and independent variables in the Variable View above,
the Data View window show look as follows:
Your two variables will be displayed in the columns based on the order you entered them into
the Variable View window (i.e., in our example, we first entered the independent
variable, group , so this appears in the first column, entitled ). Therefore, the
second column, , reflects our continuous dependent variable, coping_stress .
Now, you simply have to enter your data into the cells under each column. Remember that "each
row" represents "one case" (e.g., a case could be a single participant, so in row of our
example, our first case was a "Sedentary" with a "coping with workplace stress" (CWWS) score
of "3.18"). Since these cells will initially be empty, you need to click into the cells to enter your
data. You'll notice that when you click into the cells under your independent variable, SPSS
Statistics will give you a drop-down option with your two groups already populated. However, for
your dependent variable, you simply need to enter the values.
Running the Explore... procedure

The following instructions show you how to run the Explore... procedure in order to detect outliers
and check if your data is normally distributed:
Click Analyze > Descriptive Statistics > Explore... on the main menu, as shown below:

Transfer the dependent variable, coping_stress , into the Dependent List: box by highlighting it

(by clicking on it) and clicking the top button. Also, transfer the independent
variable, group , into the Factor List: box by highlighting it (by clicking on it) and clicking the

middle button. You will end up with the following screen:


Click the button and you will be presented with the Explore: Plots dialogue box, as
shown below:

Keep the default Factor levels together in the Boxplots area, but deselect Stem-and-leaf in the
Descriptive area and selectNormality plots with tests, so that you end up with the following
screen:
Click the button. You will be returned to the Explore dialogue box.
Click the Plots option in the Display area. This will result in only those options you selected in
the Explore: Plots dialogue box being produced. You will end up with the following screen:

Click the button to generate the output. There will be a great deal of output, including
many statistics and graphs that you will not need. The next section of this guide will explain which
particular parts of the output you need and how to interpret them.
Determining if your data has outliers

You need to look at the boxplot you just generated to determine whether there are any outliers in
any of your groups. This boxplot is shown below:

Any data points that are more than 1.5 box-lengths from the edge of their (brown-yellow) box are
classified by SPSS Statistics as outliers and are illustrated as circular dots. Any data points that are
more than 3 box-lengths away from the edge of their box are classified as extreme points (i.e.,
extreme outliers) and are illustrated with an asterisk (*). Both of these types of outliers are labelled
with their case number their row number in the Data View window for easy identification. In
this example, you can see that there are no outliers in the data, as evidenced by the lack of any
circular points or asterisks. In order for you to understand what you are looking for, an example is
shown below of what a boxplot might look like with an outlier and extreme outlier in the data:
Using boxplots in SPSS Statistics is an easy, straightforward method to detect outliers. It is not
foolproof and there other methods that are more complicated (and [arguably] more theoretically
sound). However, boxplots are demonstrated here because they are a popular and straightforward
method for detecting outliers.

Your options with regard to outliers can be divided into two camps: either keeping the outliers or
removing them, as discussed below:

Keeping the outlier(s)

If you do not want to remove an outlier or feel you cannot, you have four choices how to proceed:

1. Run the non-parametric Kruskal-Wallis H Test;


2. Modify the outlier by replacing the outlier's value with one that is less extreme (e.g., the next
largest value instead);
3. Transform the dependent variable; or
4. Include the outlier in the analysis anyway because you do not believe the result will be materially
affected (e.g., perhaps determined by comparing the result of the one-way ANOVA on the original
data to the transformed data).

Determining if your data is normally distributed

Your data can be checked to determine whether it is normally distributed using a variety of tests.
This section of the guide will concentrate on one of the most common methods: the Shapiro-Wilk
test of normality. This is a numerical method and the result of this test is available in the output
because it was run when you selected the Normality plots with tests option in the Explore:
Plots dialogue box. Other methods of determining if your data is normally distributed, such as
skewness and kurtosis values, or histograms, can be found in our Testing for Normality guide.
Whilst it is most common to run only one type of normality test for a given analysis and to rely
solely on that result, as you become more familiar with statistics you might start to evaluate
normality based on the result of more than one method. However, in this guide, we focus on a
single method (i.e., Shapiro-Wilk's test for normality).

Shapiro-Wilk test of normality

The Shapiro-Wilk test is recommended if you have small sample sizes (< 50 participants) and are
not confident visually interpreting Normal Q-Q Plots or other graphical methods. The Shapiro-
Wilk test tests if data is normally distributed for each group of the independent variable. Therefore,
there will be as many Shapiro-Wilk tests as there are groups of the independent variable. In this
example, this would mean that four tests have been run one for each group of the independent
variable, group (i.e., the "Sedentary", "Low", "Moderate" and "High" groups). Each test is
presented on a new row in the Tests of Normality table, as shown below:

In order to understand whether the scores in each group are normally distributed, you need to
consult the "Sig." column located under the "Shapiro-Wilk" column, as highlighted above. If your
data is normally distributed (i.e., the assumption of normality is met), the significance level (the
value in the "Sig." column) should be more than .05 (i.e., p > .05). If your data is not normally
distributed (i.e., the assumption of normality is violated), the significance level ("Sig.") will be
less than .05 (i.e., p < .05). The null hypothesis of the Shapiro-Wilk test is that your data's
distribution is equal to a normal distribution and the alternative hypothesis is that your data's
distribution is not equal to a normal distribution. Thus, if you reject the null hypothesis (p < .05),
this means that your data's distribution is not equal to a normal distribution.

In the table above, you can see that all of the "Sig." values are greater than .05 (they are .538, .059,
.940 and .671). Therefore, the dependent variable, coping_stress , is normally distributed for each
group of the independent variable, group .

Note: If your sample sizes are greater than 50, you would probably benefit from using graphical
methods such as a Normal Q-Q Plot because at larger sample sizes the Shapiro-Wilk test will flag
even minor deviations from normality as statistically significant (i.e., not normally distributed).
Dealing with violations of normality

If your data is not normally distributed, you have four options:

1. Transform your data.


2. Use a non-parametric test.
3. Carry on regardless.
4. Test comparisons.

Transform your data

You can transform your data so that, hopefully, your transformed data is normally distributed. You
can then run the one-way ANOVA on this transformed data. To learn how to transform your data
using the correct transformation, you can read our Transforming data guide. Remember that you
will need to re-run the tests of assumptions on the transformed data (i.e., go back to the Running
the Explore...procedure section). Transformations will generally only work when the distribution
of scores in all groups are the same shape (e.g., if all distributions are skewed to the left). Even
then, some distributions that need transforming do not have an available transformation to 'turn'
them to normality. This is particularly the case when the distributions have different shapes, such
as opposite skews, where there is not likely to be an available transformation. Alternately, for some
common distributions there are well-known transformations. Another problem is that it is generally
much harder to interpret the transformed data, which no longer represents the original values.
Use a non-parametric test

You can run a non-parametric test such as the Kruskal-Wallis H test. Although this can be a
popular alternative to the one-way ANOVA, you should be somewhat cautious because the null
and alternative hypotheses are not the same as the one-way ANOVA (see our Kruskal-Wallis H
Test guide for further information).

Carry on regardless

Run the test regardless because the one-way ANOVA is fairly "robust" to deviations from
normality, particularly if the sample sizes (numbers in each group) are equal, or nearly equal, but
less so for unequal (unbalanced) group sizes (Liz, Keselman & Keselman, 1996). Indeed, if sample
sizes are not small, even fairly skewed distributions as long as the groups are similarly skewed
are not always problematic (Sawilowsky & Blair, 1992). In conclusion, non-normality does not
affect Type I error rate substantially and the one-way ANOVA can be considered robust to non-
normality (see Maxwell & Delaney (2004) for a good review). However, if you choose this route,
you should still report the violation in your results.

Note: You should note that the "robust" aspect is with respect to a Type I error and not the power
(Type II error) of the test (Shadish et al., 2002; Wilcox, 2012), which is relatively rarely discussed.
Moreover, the literature can seem contradictory on the issue of robustness, a fact already
highlighted by some (e.g., Rutherford, 2011).

Test comparisons

This is a slightly more advanced approach. You transform the data (if that is possible) and run a
one-way ANOVA on the transformed data and on the original data. Again, if you are happy that
the conclusions reached are the same, choose the one-way ANOVA on the untransformed, original
data for analysis.

ONEWAY procedure with a post hoc test

The following instructions show you how to run a one-way ANOVA using SPSS
Statistics' ONEWAY procedure, including which options to select to generate a test of the
homogeneity of variances and a post hoc test to determine which group means differ from which
other group means. Because you will not know whether the assumption of homogeneity of
variances is met until after you have run the procedure, you will also be shown how to run a test
that allows there to be unequal variances called the Welch ANOVA and how to run a post hoc test
that also allows unequal variances. This will result in a set of parallel results: one for when the
assumption of homogeneity of variances is met and one for when it is violated. Based on the test
result for this assumption, you will be shown which results to interpret (e.g., standard one-way
ANOVA or Welch ANOVA).

To run a one-way ANOVA with a post hoc test in SPSS Statistics, follow the instructions below:

Click Analyze > Compare Means > One-Way ANOVA... on the top menu, as shown below:

You will be presented with the One-Way ANOVA dialogue box, as shown below:
Transfer the dependent variable, coping_stress , into the Dependent List: box and the independent

variable, group , into theFactor: box, using the buttons, as shown below:

Explanation: The Dependent List: box is where you enter the dependent variable(s) you want to
analyse. You can transfer more than one dependent variable into this box to simultaneously analyse
many dependent variables at the same time. The independent variable is referred to as a factor (i.e.,
a between-subjects factor) in SPSS Statistics for this procedure. Whenever you transfer a
categorical variable for example group into the Factor: box, SPSS Statistics will
automatically use all the groups in this independent variable in its calculation of the one-way
ANOVA. If you do not want this to happen you need to deselect the group(s) you do not want to
analyse before running the one-way ANOVA procedure.
Click the button and you will be presented with the One-Way ANOVA:
Options dialogue box, as shown below:

Select the Descriptive, Homogeneity of variance test and Welch checkboxes in the
Statistics area and select the Means plot checkbox. You should end up with the following screen:

Note: With the exception of the Descriptive option, the options that you have selected above will
instruct SPSS Statistics to test for homogeneity of variances using Levene's Test for Equality of
Variances, and provide a robust ANOVA (Welch's ANOVA) in case the assumption of
homogeneity of variances is violated. In addition, a means plot will be generated so that you can
get an impression of the data (e.g., detect any trends in the data).

Click the button and you will be returned to the One-Way ANOVA dialogue box.

Click the button and you will be presented with the One-Way ANOVA: Post Hoc
Multiple Comparisons dialogue box, as shown below:

The two main areas of this dialogue box reflect the two major categories of post hoc test: those
that depend on the assumption of homogeneity of variances being met and those that don't, as
illustrated below:
Select the Tukey checkbox in the Equal Variances Assumed area and the Games-
Howell checkbox in the Equal Variances Not Assumed area, as shown below:

If you want, you can change the significance level to another value (e.g., 0.01) in the Significance
level: box. In this example, keep the significance level at the default 0.05 (i.e., declare statistical
significance when p < .05).

Click the button and you will be returned to the One-Way ANOVA dialogue box.

Click the button. This will generate the output.


Interpreting the descriptive statistics

At this stage of the analysis you will have analysed your data for outliers and normality, but not
yet interpreted whether the third major assumption homogeneity of variances is violated or not.
Although this information is presented in the output you have generated, the descriptive statistics
are discussed first because these will remain the same irrespective of whether the assumption of
homogeneity of variances is met and it is good to have an overall impression of what your data is
showing.

SPSS Statistics will have generated a Descriptives table containing some useful descriptive
statistics for each group of the independent variable the "Sedentary", "Low", "Moderate" and
"High" groups which will help you get a "feel" for your data and will be used when you report
your results. You can make an initial interpretation of your data using the Descriptives table:

You might, therefore, report these results as follows:


Data is presented as mean standard deviation. The ability to cope with workplace-related stress
(CWWS score) increased from the sedentary (n = 7, 4.2 0.8), to low (n = 9, 5.9 1.7), to
moderate (n = 8, 7.1 1.6) to high (n = 7, 7.5 1.2) physical activity groups, in that order.
Assumption of homogeneity of variances

The one-way ANOVA assumes that the population variances of the dependent variable are equal
for all groups of the independent variable. If the variances are unequal, this can affect the Type I
error rate. In our example, the (population) variance for CWWS scores, coping_stress , for all
levels of group should be equal. If this is not the case, corrections can be applied to the
calculations of the one-way ANOVA so that any violation of homogeneity of variances can be
compensated for and the test remains valid.

The assumption of homogeneity of variances is tested using Levene's test of equality of variances,
which is but one way of determining whether the variances between groups for the dependent
variable are equal. The result of this test is found in the Test of Homogeneity of Variances table,
as highlighted below:

The important column of the table above is the "Sig." column, which presents the significance
value (i.e., p-value) of the test. If Levene's test is statistically significant (i.e., p < .05), you do not
have equal variances and have violated the assumption of homogeneity of variances (i.e., you have
heterogeneous variances). On the other hand, if Levene's test is not statistically significant (i.e., p >
.05), you have cannot reject the null hypothesis of the equality of variance. In our example, the
"Sig." value (i.e., p-value) is .120 (i.e., p = .120), which indicates that we cannot reject the null
hypothesis of the equality of variances.
Explanation: Levene's test for equality of variances tests the null hypothesis that the population
variances are equal or, stated another way, that the group samples are drawn from populations with
the same variance. This can be written formally as:

H0: 12 = 22 = 32 = 42
Where = population standard deviation (remembering that variance is standard deviation
squared) and the subscripts 1, 2, 3 and 4 represent the four independent groups in this example.
The alternative hypothesis of this test is that the population variances are not all equal. Again, this
can be formally written as:
HA: not all variances are equal in the population

Levene's test operates in the same way as most inferential statistical tests. In this case, it calculates
a statistic which is compared to an F-distribution, with the p-value obtained indicating the
evidence against the null hypothesis.
Results when homogeneity of variances is met

You established on the previous page that you have homogeneity of variances. This means you
can interpret the standard one-way ANOVA and, if this test is statistically significant, either: (a)
interpret the results from the Tukey post hoc test to understand where any difference(s) lie; or (b)
run contrasts to investigate specific differences between groups. The one-way ANOVA result is
found in the ANOVA table, as shown below:
The most important part of the table above is the "Sig." column, as highlighted below:

This column contains the statistical significance value (i.e., p-value) of the test found in the "Sig."
column as highlighted in the table above. If the ANOVA is statistically significant (i.e., p < .05),
it can be concluded that not all group means are equal in the population (i.e., at least one group
mean is different to another group mean). Alternatively, if p > .05, you do not have any statistically
significant differences between the group means. The p-value in this example would appear to
be .000 (obtained from the "Sig." column). However, if you ever see SPSS Statistics print out a p-
value of .000, do not interpret this as a significance value that is actually zero; it actually means p <
.0005. As the statistical significance value in this example is less than .05 (i.e., p < .0005
satisfies p < .05), it can be concluded that there is a statistically significant difference in
mean coping_stress scores for the different levels of group . That is, you know that at least one
group mean differs from the other group means. You could report this result as:
The ability to cope with workplace-related stress (CWWS score) was statistically significantly
different for different levels of physical activity group, F(3, 27) = 8.316, p < .0005.
The last part of the statement above (i.e., F(3, 27) = 8.316, p < .0005) is obtained from
the ANOVA table, as shown below:
On the other hand, if you did not find a statistically significant difference, you could report the
result as:
There were no statistically significant differences in CWWS score between the different physical
activity groups, F(3, 27) = 1.116, p = .523.
Note: The p-value used for the statement above was made up to fit the statement.

Tukey post hoc test

As has been mentioned previously, if you have no prior hypotheses about which specific groups
might differ or your interest is in all possible pairwise comparisons, you should run a post hoc test
that tests all possible group comparisons. The Tukey post hoc test is a good (Westfall et al., 2011)
and recommended (Kirk, 2013) test for this purpose when the assumption of homogeneity of
variances is not violated (and all other assumptions of the one-way ANOVA are met). This test is
useful in that it not only provides the statistical significance level (i.e., p-value) for each pairwise
comparison, but also provides confidence intervals (aka Tukey's intervals) for the mean difference
for each comparison.

The results from the Tukey post hoc test are presented in the Multiple Comparisons table, as
shown below:
It was shown earlier that with four groups in your independent variable there will be a total of six
possible combinations of group differences. However, the table above would appear to show
twelve combinations, twice the number we expect (N.B., we are referring to the table above the
note). The reason for this is that each comparison is duplicated, reflecting the two ways in which
each comparison can be calculated. That is, you could consider a comparison between two groups
(e.g., Group A vs Group B) as Group A minus Group B or, alternatively, as the reverse (i.e., Group
B minus Group A). SPSS Statistics shows both versions in the table above. For example, consider
the comparison between the "Sedentary" and "Low" groups, as highlighted below:
You will notice that in the first highlighted row, "Sedentary" is found in the first column, "(I)
group", and "Low" is found in the second column, "(J) group", whereas they are entered in an
opposite fashion in the second highlighted row. It is important to notice the "I-J" part in the next
column, "Mean Difference (I-J)". This column represents the mean difference between the two
groups in the comparison (i.e., between the "Sedentary" and "Low" groups) and is Group I minus
Group J. For the first highlighted row, this is "Sedentary" (I) minus "Low" (J). If we remind
ourselves of the mean values for the "Sedentary" and "Low" groups by consulting
the Descriptives table below:

We can see that the mean CWWS score is 4.1513 for the "Sedentary" group and 5.8789 for the
"Low" group. Thus, the first contrast we highlighted, which was "Sedentary" (I) minus "Low" (J),
is 4.1513 5.8789 = -1.7276. This is the value you find in the "Mean Difference (I-J)" column,
as highlighted below:
On the other hand, the second group comparison between the "Sedentary" and "Low" groups was
considered as "Low" (I) minus "Sedentary" (J), which is 5.8789 4.1513 = 1.7276. As you can
see, the magnitude of the mean difference is the same (as you would expect), but the direction of
the difference is different (i.e., this difference is positive, but the other was negative). They
essentially present the same information. You can see this further by consulting the "Std. Error"
and "Sig." columns, which represent the standard error of the mean difference and the statistical
significance value (i.e., the p-value) of this mean difference, respectively. These are identical for
both group comparisons, as highlighted below:
Given the examination of the two 'same' group comparisons above, which group comparison
should you interpret? The answer to this question is that it really does not matter which group
comparison you interpret as long as you take care to understand the direction of the difference, but
it can help to use the group comparison that makes the most sense to your study. For example, in
this example study, higher CWWS scores (i.e., coping_stress score) are associated with greater
durations of weekly physical activity (i.e., "High" > "Moderate" > "Low" > "Sedentary" groups
of group ) and so it might make more sense if positive mean differences were used (i.e., lower
scores subtracted from the higher scores) rather than the opposite. This means we are interested in
the "(I) Group" column containing the higher physical activity level groups compared to the "(J)
Group" column, as highlighted below:
Let us now interpret the comparison between the "Sedentary" and "Low" groups, as highlighted
below:

This comparison states that the mean CWWS score in the "Low" group is 1.72762 higher than the
"Sedentary" group. To determine whether this mean difference is statistically significant, you need
to consult the "Sig." column, which presents the p-value. For this comparison, the p-value
is .092 (i.e., p = .092). If p < .05, you have a statistically significant result, but if p > .05, you do
not have a statistically significant result. For our comparison between the "Sedentary" and "Low"
groups, p = .092, which is greater than .05 and, therefore, the mean difference between these two
groups is not statistically significant (i.e., the mean difference is not different from zero in the
population).
As such, if you add in information from the Descriptives table (see above or the Descriptive
statistics section earlier) in addition to the information provided by Tukey's post hoc test results
from the Multiple Comparisons table, you could report this comparison as follows:
There was an increase in CWWS score from 4.2 0.8 in the sedentary group to 5.9 1.7 in the
group performing a low level of physical activity, an increase of 1.7 (95% CI, -0.2 to 3.7), which
was not statistically significant (p = .092).
Let us now consider the comparison between the "High" and "Sedentary" groups, as highlighted
below:

We can see that the difference between these two groups is approximately twice that of our
previous comparison with the "High" group having a CWWS score 3.35409 higher than the
"Sedentary" group (the "Mean Difference (I-J)" column), which is a statistically significant mean
difference, p = .001 (the "Sig." column). You could report this result as follows:
There was an increase in CWWS score from 4.2 0.8 in the sedentary group to 7.5 1.2 in the
group performing a high level of physical activity, an increase of 3.4 (95% CI, 1.3 to 5.4), which
was statistically significant (p = .001).
If you do not have homogeneity of variances across groups you cannot interpret the standard one-
way ANOVA, but must use a modified version of the ANOVA. In this example, the Welch
ANOVA is used. If this test is statistically significant, you can either: (a) interpret the results of
the Games-Howell post hoc test to understand where any difference(s) lie; or (b) run contrasts to
investigate specific differences between groups. The result of the Welch's ANOVA is found in
the Robust Tests of Equality of Means table, as shown below:

The most important part of the table above is the "Sig." column, as highlighted below:

You might also like