ANCOVA and MANCOVA: Example Questions and Outline Model Answers

Question 1
A double-blind, placebo-controlled trial was carried out to test a new drug which is supposed to improve episodic memory in healthy elderly
subjects, many of whom show mild episodic memory impairment. Before starting the trial the verbal IQ of the subjects was assessed. The
subjects were then assigned at random to an active drug or placebo treatment. After completing the treatment the researchers administered 3
standard verbal episodic memory tests to each subject. They felt that each test was measuring the memory processes that the drug was likely to
improve. After breaking the drug codes, the researchers used ANOVAs to see if the post-treatment memory scores were higher in the drugtreated subjects. The results are shown in Part I of the SPSS printout. Disappointed that only one memory test appeared to show a significant
drug effect, the researchers consulted colleagues. After looking at the correlations (printout Part II) between the memory test performance and
pretreatment verbal IQ, one colleague suggested that they repeat the analyses covarying out the verbal IQ scores, stressing that they should test
the homogeneity of regressions first. The homogeneity of regression and ANCOVAs (printouts Part III and IV respectively) were as shown.
(a)

Explain what the homogeneity of regression printout shows and say, in light of these results, whether the ensuing ANCOVAs were
justified. (30% of marks for question)

(b)

Briefly explain the purpose of the ANCOVAs and why the ANCOVA results show stronger drug effects than the earlier ANOVAs. (35% of
marks for question)

(c)

The researchers concluded that 2 of their 3 memory tests were significantly improved by the drug after removing the effect of baseline
verbal IQ. What problem means that they were not justified in making this claim? (20% of marks for question)

(d)

Another colleague suggested an alternative, but related, analytic method which avoided the problem in (c). This technique showed a
significant drug effect on episodic memory test performance after removing the effect of baseline verbal IQ. Name the technique and
briefly note why it is preferable here? (15% of marks for question)

Printout for Question 1

Part I Anova Results + Means

Tests of Between-Subjects Effects
Source
TREATMNT

Error

Dependent Variable
Memory test 1
Memory test 2
Memory test 3
Memory test 1
Memory test 2
Memory test 3

Type III Sum

of Squares
16.240
15.123
7.516
294.049
151.702
447.413

df
1
1
1
58
58
58

Mean Square
16.240
15.123
7.516
5.070
2.616
7.714

Part II Correlation Matrix

Correlations

Memory test 1

Memory test 2

Memory test 3

Pearson Correlation
Sig. (2-tailed)
N
Pearson Correlation
Sig. (2-tailed)
N
Pearson Correlation
Sig. (2-tailed)
N

Memory test 2
.434
.001
60

Memory test 3
.758
.000
60
.344
.007
60

Verbal IQ
.652
.000
60
.388
.002
60
.353
.006
60

F
3.203
5.782
.974

Sig.
.079
.019
.328

Part III Homogeneity of Regression Results

Tests of Between-Subjects Effects
Source
VERBIQ

Dependent Variable
Memory test 1
Memory test 2
Memory test 3
TREATMNT * VERBIQ Memory test 1
Memory test 2
Memory test 3
Error
Memory test 1
Memory test 2
Memory test 3

Type III Sum

of Squares
127.211
26.396
55.744
4.338E-03
3.475
.154
164.100
123.962
391.240

df
1
1
1
1
1
1
56
56
56

Mean Square
127.211
26.396
55.744
4.338E-03
3.475
.154
2.930
2.214
6.986

F
43.412
11.925
7.979
.001
1.570
.022

Sig.
.000
.001
.007
.969
.215
.882

Part IV Ancova Results and Covariate-Adjusted Means

Tests of Between-Subjects Effects
Source
VERBIQ

TREATMNT

Error

Dependent Variable
Memory test 1
Memory test 2
Memory test 3
Memory test 1
Memory test 2
Memory test 3
Memory test 1
Memory test 2
Memory test 3

Type III Sum

of Squares
129.945
24.265
56.018
14.396
14.337
6.691
164.104
127.437
391.394

df
1
1
1
1
1
1
57
57
57

Mean Square
129.945
24.265
56.018
14.396
14.337
6.691
2.879
2.236
6.867

F
45.135
10.853
8.158
5.000
6.413
.974

Treatment Condition

Dependent Variable
Memory test 1
Memory test 2
Memory test 3

Treatment Condition
active drug
placebo
active drug
placebo
active drug
placebo

Mean
Std. Error
8.217a
.310
7.237a
.310
12.316a
.273
a
11.338
.273
9.190a
.478
a
8.522
.478

a. Evaluated at covariates appeared in the model: Verbal IQ = 92.68.

95% Confidence Interval

Lower Bound
Upper Bound
7.597
8.838
6.617
7.858
11.770
12.863
10.792
11.885
8.232
10.148
7.564
9.480

Sig.
.000
.002
.006
.029
.014
.328

Question1: MODEL ANSWER

Note how short the perfect answer can be
(a)

(b)

(c)
(d)

The homogeneity of regression printout shows non-significant covariate(IQ)*treatment group interactions for each of the 3 memory tests
used. This means that the regressions of IQ on the dependent variable (i.e. each of the memory test scores) are statistically indistinguishable
across the levels of the group (i.e. treatment) factor. (Students might draw diag with parallel regression lines to illustrate this -- must be
properly labelled; they do not need to draw a diag if their verbal description is complete and clear.) Thus the regressions are homogeneous
and mean that ANCOVAs requirement for homogeneous regressions is satisfied for each of the 3 DVs. In terms of this requirement it is
legitimiate to carry out the 3 ANCOVAs (on each of the memory tests as separate DVs) on these data.
The purpose of the ANCOVA is to reduce the size of the error term in the analyses thereby increasing power. It achieves this by removing a
source of variation from the DV (the DVs covariance with the covariate). This source of variation is normally included in the error term of
the ANOVA. The F-ratio of the ANOVA (or ANCOVA) is formed from an effect term divided by the error term: if the error term is reduced
by assigning some of its variance to a covariate, then the F-ratio for the effect of interest increases commensurately. [A student who says
that the ANCOVA is to remove any IQ differences between the drug-treatment groups gets ABSOLUTELY ZERO marks for this part.] For
memory test 1, which has by far the strongest correlation with IQ (r>0.65), we can see this in operation ANOVA: MS(effect)=16.2;
MS(error)=5.070. ANCOVA: MS(effect)=14.4; MS(error)=2.879. For the other two memory tests (with lower correlations with IQ; r<0.4),
the effect of covariate adjustment was very modest, barely affecting the F-ratios.
The researchers have capitalised on chance by conducting 3 separate tests of their hypothesis. None of these individual tests will therefore
truly have a Type I error rate of 5% and so cannot be judged by this criterion. If one adopted a Bonferroni correction to deal with this, the
necessary alpha level would be 0.05/3. (i.e. 0.0167) and only memory test 2 (after covariate adjustment) would be deemed significant.
MANCOVA (would accept MANOVA; although strictly it is MANCOVA if IQ is included as a covariate). Basically, it is preferable
because it uses all 3 memory test DVs simultaneously to carry out a single test of the researchers hypothesis rather than three separate
tests. It is appropriate because the 3 tests are thought to be measuring the same processes and the correlations between the 3 DVs are
neither close to zero or 1 (they were 0.34; 0.43; 0.76).