REGRESSION ANALYSIS DESIGNS

- p.
1/20
Statistics 203: Introduction to Regression
and Analysis of Variance
Experimental design
Jonathan Taylor
qToday
qWhy is design important?
qWhat makes a good
experiment?
qPower in a multivariate setting
qPower example: one-way
ANOVA
qDetermining sample size:
power
qScheffs procedure
qDetermining sample size: CIs
qClassical designs
qRandomized design
qRandomized block design
qNested designs
qNested design: ANOVA table
qLatin square
qLatin square ANOVA table
q2
k
factorial designs
qFractional design: example
qDesign criteria
- p. 2/20
Today
s
Experimental design in a (small) nutshell.
s
Power.
s
Classical designs.
s
Criteria of optimality.
qToday
qWhat makes a good
experiment?
ANOVA
power
qScheffs procedure
qClassical designs
qRandomized design
qNested designs
qLatin square
q2
k
factorial designs
qDesign criteria
- p. 3/20
Why is design important?
s
Entire courses based just on design: only a brief overview
today.
s
Industrial experiments. Often each trial can be very
expensive: imagine modelling crash test data for Jaguar....
s
Clinical trials. To rule out unexpected selection bias, it is
important to sufciently randomize the study, perhaps
keeping some control over heterogeneity.
s
A topic of active research: eld has progressed well beyond
its origins in agricultural eld trials. Modern response surface
design problems are used frequently in industry.
s
Current examples: CFD (Computational Fluid Dynamics)
models take ages to run on a computer and one might want
to understand how the model depends on the parameters
input into it: try a few values of the parameters and
interpolate (i.e. run a regression).
qToday
qWhat makes a good
experiment?
ANOVA
power
qScheffs procedure
qClassical designs
qRandomized design
qNested designs
qLatin square
q2
k
factorial designs
qDesign criteria
- p. 4/20
What makes a good experiment?
s
Is the design subject to unforeseen (selection) bias?
s
Is it powerful enough to detect a given effect size?
s
Is the precision of a certain number of estimators sufcient?
i.e. if we assume we know
2
, can we get a tight enough
condence interval for
1
, say?
s
Multivariate generalizations: D-optimality, V -optimality.
qToday
qWhat makes a good
experiment?
ANOVA
power
qScheffs procedure
qClassical designs
qRandomized design
qNested designs
qLatin square
q2
k
factorial designs
qDesign criteria
- p. 5/20
Power in a multivariate setting
s
To talk about power in a multivariate setting, one needs to
know about non-central
2
, F, t.
s
Non-central
2
: suppose Z N(, I) R
k
. Then
Z
2

2
k
(
2
).
and
2
is called the non-centrality parameter: 0
corresponds to usual
2
k
.
s
Non-central F: G
1

2
1
(h), G
2

2
2
(0) then
F =
G
1
/
1
G
2
/
2
F
1
,
2
(h).
s
Non-central t
k
(h): square root of F
1,k
(h
2
).
qToday
qWhat makes a good
experiment?
ANOVA
power
qScheffs procedure
qClassical designs
qRandomized design
qNested designs
qLatin square
q2
k
factorial designs
qDesign criteria
- p. 6/20
Power example: one-way ANOVA
s
Source SS df E(MS)
Treatments SSTR =
P
r
i=1
n
i
Y
i
Y
2
r 1
2
+
P
r
i=1
n
i
2
i
r1
Error SSE =
P
r
i=1
Pn
i
j=1
(Y
ij
Y
i
)
2
P
r
i=1
n
i
r
2
s
Power for testing H
0
:
1
= =
r
= 0. Under
H
a
:
r
i=1
n
i
2
i
= h
MSTR
MSE
F
df
TR
,df
E
()
where
=
i
n
i
2
i
(r 1)
2
=
h
(r 1)
2
.
Therefore, if we reject at level
Power = 1 P
df
TR
,df
E
,
(F
df
TR
,df
E
,1
).
where P
1
,
2
,
is the distribution function of F
1
,
2
()
qToday
qWhat makes a good
experiment?
ANOVA
power
qScheffs procedure
qClassical designs
qRandomized design
qNested designs
qLatin square
q2
k
factorial designs
qDesign criteria
- p. 7/20
Determining sample size: power
s
Suppose that n
i
= n, then the non-centrality parameter
> (max
i
i
min
i
i
)
2
n
(r 1)
2
=

s
As power is monotone increasing in , it is possible to just
specify this range of s relative to in the form of

.
s
To determine sample size for a given power 1 and Type I
error rate we would plot Power as a function of sample
size and take the smallest sample size that yields sufcient
power.
s
Here is the example.
qToday
qWhat makes a good
experiment?
ANOVA
power
qScheffs procedure
qClassical designs
qRandomized design
qNested designs
qLatin square
q2
k
factorial designs
qDesign criteria
- p. 8/20
Scheffs procedure
s
We saw Bonferroni correction for simultaneous inference for
residuals.
s
Suppose we wanted condence intervals for many contrasts
p
j=0
a
j
j
.
For instance, all pairwise differences of main effects in a
two-way ANOVA model.
s
We can use Bonferroni, but as the number of contrasts grows
intervals get wider even though they are based on only p
random variables: cannot use this to get condence bands.
s
Scheffs procedure: condence interval for each contrast is
p
j=0
a
j
j
SE
_
_
p
j=0
a
j
j
_
_
_
pF
p,np,1
.
qToday
qWhat makes a good
experiment?
ANOVA
power
qScheffs procedure
qClassical designs
qRandomized design
qNested designs
qLatin square
q2
k
factorial designs
qDesign criteria
- p. 9/20
Determining sample size: CIs
s
Suppose that there is a set of k contrasts that we wish to
estimate and each one has a pre-specied target width w
i
.
s
Scheffs procedure at level tells us that the CI for the j-th
contrast of interest
r
j=1
a
i,j
j
has width
W
i
= 2
2
(rF
r,(n1)r,1
)
r
j=1
a
2
i,j
n
s
Can replace rF
r,(n1)r,1
by Bonferroni correction
t
2
(n1)r,1/k
.
s
Choose n large enough so that W
i
w
i
, 1 i k.
qToday
qWhat makes a good
experiment?
ANOVA
power
qScheffs procedure
qClassical designs
qRandomized design
qNested designs
qLatin square
q2
k
factorial designs
qDesign criteria
- p. 10/20
Classical designs
s
Randomized.
s
Randomized complete block.
s
Nested.
s
Repeated measures will come later in random effects.
s
Latin square.
s
Factorial / fractional factorial.
qToday
qWhat makes a good
experiment?
ANOVA
power
qScheffs procedure
qClassical designs
qRandomized design
qNested designs
qLatin square
q2
k
factorial designs
qDesign criteria
- p. 11/20
Randomized design
s
This design controls selection bias in the experiment: i.e. by
assigning tter people to treatment vs. control, looks like
treatment is more effective than it is.
s
Given r treatments, and nr subjects, assign subjects to
treatment are random.
s
Assumes implicitly that all subjects are identical no
controlling for variables such as gender, age, etc.
s
Reduces to a one-way ANOVA model for the treatment
effects.
Y
ij
=
+
i
+
ij
, 1 i r, 1 j n
(with usual constraints)
qToday
qWhat makes a good
experiment?
ANOVA
power
qScheffs procedure
qClassical designs
qRandomized design
qNested designs
qLatin square
q2
k
factorial designs
qDesign criteria
- p. 12/20
Randomized block design
s
If subjects are heterogeneous then some of the variance
2
can be attributed to this heterogeneity.
s
One may block subjects into n homogeneous groups and
randomize the r treatments within each block.
s
Reduces to a two-way ANOVA model for the block and
treatment effects with no interactions.
Y
ij
=
+
i
+
j
+
ij
s
SS df E(MS)
SSBL = r
P
i
Y
i
Y
2
n 1
2
+ r
P
i

2
i
r1
SSTR = n
P
j
Y
j
Y
2
r 1
2
+ n
P
j

2
j
r1
SSBL.TR = r
P
i,j
Y
ij
Y
i
Y
j
+ Y
2
(n 1)(r 1)
2
qToday
qWhat makes a good
experiment?
ANOVA
power
qScheffs procedure
qClassical designs
qRandomized design
qNested designs
qLatin square
q2
k
factorial designs
qDesign criteria
- p. 13/20
Nested designs
s
Example: suppose we are studying the performance of
different schools on standardized tests based on the
performance of classes within the schools.
s
Each school 1 i a has b classes taking the tests, of
which each class had a different teacher.
s
It is natural to think of school effect and teacher effect, but
the teachers taught only within one school: they are nested
within schools. (Perhaps better to treat this as random ...)
qToday
qWhat makes a good
experiment?
ANOVA
power
qScheffs procedure
qClassical designs
qRandomized design
qNested designs
qLatin square
q2
k
factorial designs
qDesign criteria
- p. 14/20
Nested design: ANOVA table
s
Model: like a two-way ANOVA model
Y
ijk
=
+
i
+
j(i)
+
ijk
with 1 i a, 1 j b, 1 k n.
s
Note the
j(i)
s are not shared across schools and can only
be estimated within a given school.
s
SS df E(MS)
SSA = bn
P
i
Y
i
Y
2
a 1
2
+ bn
P
i

2
i
a1
SSB(A) = n
P
i,j
Y
ij
Y
i
2
a(b 1)
2
+ n
P
i,j

2
j(i)
a(b1)
SSE =
P
i,j,k
Y
ijk
Y
ij
2
ab(n 1)
2
qToday
qWhat makes a good
experiment?
ANOVA
power
qScheffs procedure
qClassical designs
qRandomized design
qNested designs
qLatin square
q2
k
factorial designs
qDesign criteria
- p. 15/20
Latin square
s
r treatments: two blocking variables: each block gets all r
treatments
s
Example:
Time
Subject
A B C
B C A
C A B
s
Model
Y
ijk
=
...
+
i
+
j
+
k
+
ijk
1 i, j, k r
but only r
2
observations.
s
Similar to two-way ANOVA model with no interactions, one
replication per cell.
s
Y
i
=
1
r
j
Y
ijk
, Y
j
=
1
r
i
Y
ijk
, Y
k
=
1
r
i,j
Y
ijk
.
qToday
qWhat makes a good
experiment?
ANOVA
power
qScheffs procedure
qClassical designs
qRandomized design
qNested designs
qLatin square
q2
k
factorial designs
qDesign criteria
- p. 16/20
Latin square ANOVA table
s
Predicted values
Y
ijk
= Y
i
+Y
j
+Y
k
2Y
s
SS df E(MS)
SSROW = r
i
_
Y
i
Y
_
2
r 1
2
+r
P
i

2
i
r1
SSCOL = r
j
_
Y
j
Y
_
2
r 1
2
+r
P
j

2
j
r1
SSTR = r
k
_
Y
k
Y
_
2
r 1
2
+r
P
k

2
k
r1
SSRem =
i,j,k
(Y
ijk

Y
ijk
)
2
(r 1)(r 2)
2
qToday
qWhat makes a good
experiment?
ANOVA
power
qScheffs procedure
qClassical designs
qRandomized design
qNested designs
qLatin square
q2
k
factorial designs
qDesign criteria
- p. 17/20
2
k
factorial designs
s
Given k factors with 2 levels each there are 2
k
possible
combinations. Designs including all levels are called 2
k
factorial designs.
s
To estimate all of them (with replications) becomes quite
expensive.
s
Most of the degrees of freedom goes to estimating higher
order interactions (which may be less of interest).
s
A 2
kf
design is a design that drops some combinations in
the interest of cost, but introduces some confounding to the
model.
s
For instance if f = 1 then the experimenter only uses half of
the combinations. This means that it may be impossible to
separate some low order interactions effects with higher
order interactions (i.e. they will be confounded)
s
Which effects are confounded depends on the dening
relation of the fractional study. Good dening relations leave
as many low order interactions (including main effects)
estimable.
qToday
qWhat makes a good
experiment?
ANOVA
power
qScheffs procedure
qClassical designs
qRandomized design
qNested designs
qLatin square
q2
k
factorial designs
qDesign criteria
- p. 18/20
Fractional design: example
s
Three factors: full design
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
qToday
qWhat makes a good
experiment?
ANOVA
power
qScheffs procedure
qClassical designs
qRandomized design
qNested designs
qLatin square
q2
k
factorial designs
qDesign criteria
- p. 19/20
Fractional design: example
s
Three factors: half design
_
_
_
_
_
1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1
_
_
_
_
_
qToday
qWhat makes a good
experiment?
ANOVA
power
qScheffs procedure
qClassical designs
qRandomized design
qNested designs
qLatin square
q2
k
factorial designs
qDesign criteria
- p. 20/20
Design criteria
s
In incomplete fractional designs, it gets very tricky to sort out
what is confounded with what.
s
Need for general criteria to compare designs.
s
D-criterion
D = det(X
t
X)
s
V -criterion: given a collection {X
1
, . . . , X
k
} of points of
interest in the predictor space
V =
1
k
k
i=1
X
i
(X
t
X)
1
X
t
i
.

REGRESSION ANALYSIS DESIGNS

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

REGRESSION ANALYSIS DESIGNS

Uploaded by

Copyright:

Available Formats

- p.

You might also like