Fixed Random

Statistics 203: Introduction to Regression and Analysis of Variance
Fixed vs. Random Effects

Jonathan Taylor
- p. 1/19
Todays class
q Todays class q Two-way ANOVA q Random vs. xed effects q When to use random effects? q Example: sodium content in beer q One-way random effects model q Implications for model q One-way random ANOVA table q Inference for q Estimating
s s s s
Random effects. One-way random effects ANOVA. Two-way mixed & random effects ANOVA. Sattherwaites procedure.
q Example: productivity study q Two-way random effects model q ANOVA tables: Two-way (random) q Mixed effects model q Two-way mixed effects model q ANOVA tables: Two-way (mixed) q Condence intervals for variances q Sattherwaites procedure
- p. 2/19
Two-way ANOVA
s s
Second generalization: more than one grouping variable. Two-way ANOVA model: observations: (Yijk ), 1 i r, 1 j m, 1 k nij : r groups in rst grouping variable, m groups ins second and nij samples in (i, j )-cell: Yijk = + i + j + ( )ij + ijk , ijk N (0, 2 ).
Constraints: r x i=1 i = 0 m x j =1 j = 0 m x j =1 ( )ij = 0, 1 i r r x i=1 ( )ij = 0, 1 j m.
- p. 3/19
Random vs. xed effects
s s
In ANOVA examples we have seen so far, the categorical variables are well-dened categories: below average tness, long duration, etc. In some designs, the categorical variable is subject. Simplest example: repeated measures, where more than one (identical) measurement is taken on the same individual. In this case, the group effect i is best thought of as random because we only sample a subset of the entire population of subjects.
- p. 4/19
When to use random effects?
A group effect is random if we can think of the levels we observe in that group to be samples from a larger population. Example: if collecting data from different medical centers, center might be thought of as random. Example: if surveying students on different campuses, campus may be a random effect.
- p. 5/19
Example: sodium content in beer
How much sodium is there in North American beer? How much does this vary by brand? Observations: for 6 brands of beer, researchers recorded the sodium content of 8 12 ounce bottles. Questions of interest: what is the grand mean sodium content? How much variability is there from brand to brand? Individuals in this case are brands, repeated measures are the 8 bottles.
- p. 6/19
One-way random effects model
s s s s s
Suppose we take n identical measurements from r subjects. Yij + i + ij , 1 i r, 1 j n ij N (0, 2 ), 1 i r, 1 j n

2 i N (0, ), 1 i r.
We might be interested in the population mean, : CIs, is it zero? etc. Alternatively, we might be interested in the variability across 2 : CIs, is it zero? subjects,
- p. 7/19
Implications for model
In random effects model, the observations are no longer independent (even if s are independent). In fact
2 i,i + 2 j,j . Cov(Yij , Yi j ) =
In more complicated mixed effects models, this makes MLE more complicated: not only are there parameters in the mean, but in the covariance as well. In ordinary least squares regression, the only parameter to estimate is 2 because the covariance matrix is 2 I .
- p. 8/19
One-way random ANOVA table
Source Treatments Error
Only change here is the expectation of SST R which reects randomness of i s. ANOVA table is still useful to setup tests: the same F statistics for xed or random will work here. 2 Under H0 : = 0, it is easy to see that M ST R Fr1,(n1)r . M SE
SS 2 Pr n SST R = Y Y i i=1 Pr Pn 2 SSE = i=1 j =1 (Yij Y i )
d f r1 (n 1)r
E (M S ) 2 2 + n 2
- p. 9/19
Inference for
We know that E (Y ) = , and can show that

2 n + 2 . Var(Y ) = rn
Therefore, Y
SST R (r 1)rn
tr1
Why r 1 degrees of freedom? Imagine we could record an innite number of observations for each individual, so that Y i i . To learn anything about we still only have r observations (1 , . . . , r ). Sampling more within an individual cannot narrow the CI for .
- p. 10/19
2 Estimating
q Todays class q Two-way ANOVA q Random vs. xed effects q When to use random effects? q Example: sodium content in beer q One-way random effects model q Implications for model q One-way random ANOVA table q Inference for q Estimating 2
From the ANOVA table

2
E (SST R/(r 1)) E (SSE/((n 1)r)) . = n SST R/(r 1) SSE/((n 1)r) = n
Natural estimate:
2 S
Problem: this estimate can be negative! One of the difculties in random effects model.
- p. 11/19
Example: productivity study
Imagine a study on the productivity of employees in a large manufacturing company. Company wants to get an idea of daily productivity, and how it depends on which machine an employee uses. Study: take m employees and r machines, having each employee work on each machine for a total of n days. As these employees are not all employees, and these machines are not all machines it makes sense to think of both the effects of machine and employees (and interactions) as random.
- p. 12/19
Two-way random effects model
s s s s s
Yijk + i + j + ( )ij + ij , 1 i r, 1 j m, 1 kn ijk N (0, 2 ), 1 i r, 1 j m, 1 k n

2 i N (0, ), 1 i r. 2 j N (0, ), 1 j m. 2 ( )ij N (0, ), 1 j m, 1 i r.
Cov(Yijk , Yi j
2 2 2 ) = ii + jj + ii jj + ii jj kk 2 .
- p. 13/19
ANOVA tables: Two-way (random)
SS 2 P SSA = nm r Y Y i i=1 2 Pm SSB = nr j =1 Y j Y 2 P Pm Y Y Y + Y SSAB = n r i j ij i=1 j =1 Pm Pn Pr 2 SSE = j =1 i=1 k=1 (Yijk Y ij )
d f r1 m1 (m 1)(r 1) (n 1)ab
E (SS ) 2 + n 2 2 + nm 2 2 2 + nr + n 2 2 + n 2
s s
2 To test H0 : = 0 use SSA and SSAB . 2 To test H0 : = 0 use SSAB and SSE .
- p. 14/19
Mixed effects model
q Todays class q Two-way ANOVA q Random vs. xed effects q When to use random effects? q Example: sodium content in beer q One-way random effects model q Implications for model q One-way random ANOVA table q Inference for q Estimating 2
In some studies, some factors can be thought of as xed, others random. For instance, we might have a study of the effect of a standard part of the brewing process on sodium levels in the beer example. Then, we might think of a model in which we have a xed effect for brewing technique and a random effect for beer.
- p. 15/19
Two-way mixed effects model
s s s s s
Yijk + i + j + ( )ij + ij , 1 i r, 1 j m, 1 kn ijk N (0, 2 ), 1 i r, 1 j m, 1 k n

2 i N (0, ), 1 i r. j , 1 j m are constants. 2 ( )ij N (0, (m 1) /m), 1 j m, 1 i r.
Constraints: m x j =1 j = 0 r x i=1 ( )ij = 0, 1 i r. x Cov (( )ij , ( )i j ) = 2 /m Cov(Yijk , Yi j jj

2 + ii k
)=
1 2 (1 ii ) m + ii kk 2
m1 2 m
- p. 16/19
ANOVA tables: Two-way (mixed)
SS SSA SSB SSAB Pm Pn Pr 2 SSE = j =1 i=1 k=1 (Yijk Y ij )
d f r1 m1 (m 1)(r 1) (n 1)ab
E (M S ) 2 2 + nm Pm 2 j =1 i 2 + n 2 + nr m1 2 2 + n 2
s s s
2 To test H0 : = 0 use SSA and SSE . To test H0 : 1 = = m = 0 use SSB and SSAB . 2 To test H0 : use SSAB and SSE .
- p. 17/19
Condence intervals for variances
2 Consider estimating in the two-way random effects ANOVA. A natural estimate is 2 = nr(M SB M SAB ).
s s s
What about CI? A linear combination of 2 but not 2 . 2 To form a condence interval for we need to know distribution of a linear combination of M S s, at least approximately.
- p. 18/19
Sattherwaites procedure
Given k independent M S s
k
L
i=1 s
ci M S i
Then dfT L E(L) where dfT =

k i=1 ci M Si
2 dfT .
2
k 2 2 i=1 ci M Si /dfi
where dfi are the degrees of freedom of the i-th M S .

s
(1 ) 100% CI for E(L): dfT L , LL = 2 dfT ;1/2 dfT L LU = 2 dfT ;/2
- p. 19/19

Fixed Random

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Fixed Random

Uploaded by

Copyright:

Available Formats

Statistics 203: Introduction to Regression and Analysis of Variance

Fixed vs. Random Effects

Constraints: r x i=1 i = 0 m x j =1 j = 0 m x j =1 ( )ij = 0, 1 i r r x i=1 ( )ij = 0, 1 j m.

Random vs. xed effects

When to use random effects?

Example: sodium content in beer

One-way random effects model

Suppose we take n identical measurements from r subjects. Yij + i + ij , 1 i r, 1 j n ij N (0, 2 ), 1 i r, 1 j n

Implications for model

One-way random ANOVA table

Source Treatments Error

SS 2 Pr n SST R = Y Y i i=1 Pr Pn 2 SSE = i=1 j =1 (Yij Y i )

We know that E (Y ) = , and can show that

From the ANOVA table

E (SST R/(r 1)) E (SSE/((n 1)r)) . = n SST R/(r 1) SSE/((n 1)r) = n

Example: productivity study

Two-way random effects model

Yijk + i + j + ( )ij + ij , 1 i r, 1 j m, 1 kn ijk N (0, 2 ), 1 i r, 1 j m, 1 k n

ANOVA tables: Two-way (random)

SS 2 P SSA = nm r Y Y i i=1 2 Pm SSB = nr j =1 Y j Y 2 P Pm Y Y Y + Y SSAB = n r i j ij i=1 j =1 Pm Pn Pr 2 SSE = j =1 i=1 k=1 (Yijk Y ij )

Mixed effects model

Two-way mixed effects model

Yijk + i + j + ( )ij + ij , 1 i r, 1 j m, 1 kn ijk N (0, 2 ), 1 i r, 1 j m, 1 k n

Constraints: m x j =1 j = 0 r x i=1 ( )ij = 0, 1 i r. x Cov (( )ij , ( )i j ) = 2 /m Cov(Yijk , Yi j jj

ANOVA tables: Two-way (mixed)

SS SSA SSB SSAB Pm Pn Pr 2 SSE = j =1 i=1 k=1 (Yijk Y ij )

Condence intervals for variances

Then dfT L E(L) where dfT =

where dfi are the degrees of freedom of the i-th M S .

(1 ) 100% CI for E(L): dfT L , LL = 2 dfT ;1/2 dfT L LU = 2 dfT ;/2

You might also like