Statistical tests for

replicated experiments
Normal probability plots are a less
formal diagnostic tool for detecting
effects
F-tests and t-tests provide a
statistical test of factor effects
 Statistical tests for
replicated experiments
Statistical tests are possible for
unreplicated designs (unreplicated
pilot studies are essential tools in
sample size calculations)
We will first focus on statistical tests
for replicated designs
 Statistical tests for
replicated experiments--
Example
Response--Pulse rate of subject
Factors
Treatment (Energy Drink, Placebo)
Setting (Moderate, Difficult)
Machine (Stair climber, Recumbent bike)
 Statistical tests for
replicated experiments--
T S Example
M Responses Mean Variance

-1 -1 -1 109, 112, 116, 112 112.25 8.25

1 -1 -1 120, 123, 125, 120 122 6

-1 1 -1 130, 118, 118, 118 121 36

1 1 -1 125, 123, 130, 124 125.5 9.67

-1 -1 1 108, 110, 113, 111 110.5 4.33

1 -1 1 121, 123, 123, 118 121.25 5.58

-1 1 1 113, 117, 122, 119 117.75 14.21

1 1 1 127, 126, 125, 125 125.75 0.917

 Statistical tests for
replicated experiments--
Example
 Statistical tests for
replicated experiments--
Example
Normal Probability Plot of the Standardized Effects
(response is Pulse, Alpha = .05)
99
Effect Type
Not Significant
95 Significant

90 A Factor Name
A A
80 B B
B C C
70
Percent

60
50
40
30
20

10

1
-2 -1 0 1 2 3 4 5 6 7
Standardized Effect
 Statistical tests for
replicated experiments
Effect sizes depend on the
measurement scale
Statistical tests are based on
standardized effects
To compute standardized effects,
start with an estimate of
experimental error
 Statistical tests for
replicated experiments
Experimental error can be
summarized by the square root of
the variance of the background
noise (the standard deviation)
The experimental error measures
variation in a single observation
 Statistical tests for
replicated experiments
Thevariance is best estimated by
the Mean Square for Pure Error
(MSPE)

(Sum of cell variances )

MSPE
8
 Statistical tests for
replicated experiments--
Example

8.25 6 14.21 .92

MSPE 10.62
8

MSPE 3.26
Thestandard deviation for each run
is ~3 beats per minute
 Statistical tests for
replicated experiments
While the standard deviation for a
single response is the square root of
MSPE, the standard deviation of an
effect (its standard error) is:

MSPE
k 2
2 n
 Statistical tests for
replicated experiments
We divide an effect in a k-factor
experiment with n replications (e.g.,
A) by its standard error to compute a
t-test statistic :
Effect
T
MSPE
k 2
2 n
 Statistical tests for
replicated experiments
Test statistics for other effects are
computed similarly
U-do-it: Calculate the T-statistics of
all effects for the Exercise data
 Statistical tests for
replicated experiments
When an effect is negligible, T has a
t-distribution
The shape of the t-distribution curve
depends on the number of replicates
(degrees of freedom=2k(n-1))
The t-distribution curves have
slightly more spread than the bell-
shaped (normal) curve
 Statistical tests for
replicated experiments--t
curve for 3-factor design
0.4 4 reps
2 reps

0.3

0.2

0.1 2 replications

4 replications
0.0

-5 -4 -3 -2 -1 0 1 2 3 4
Statistics tests for replicated
experiments
If|T| is larger than the 99.5th or 97.5th
percentile of the t distribution, an
effect is significant
These percentiles are commonly
found in textbooks (but please use a
computer package instead)
 Statistical tests for
replicated experiments--t
critical value for 3-factor
design (n=4)

0.4

t dist. with 3 reps (24 df)

0.3

0.2
.025

0.1

2.064
0.0

-4 -3 -2 -1 0 1 2 3 4
 Statistical tests for
replicated experiments
Sometimes, twice the area to the right
of |T| is reported as a p-value. Small
p-values suggest that a standardized
effect is distinguishable from
background noise
You definitely need a computer to
compute p-values--in the following
example, the p-value for the M effect
is 2*.122=.244
Statistical test for replicated
experiments--Example
 Statistical tests for
replicated experiments--
Example
U-do-it: Compute p-values for the
remaining effects. Which effects are
significant? Are these the same
effects that the probability plot
detected?
 Statistical tests for
replicated experiments
F tests for individual effects are
equivalent to t-tests
F tests allow several comparisons to
be tested simultaneously
The t-test can be used to help in
computing the number of replications
needed in a factorial experiment
 Statistical tests for
replicated experiments
Hypothesistesting can be extended
to combine estimates of error from
both pure error and negligible effects
Negligible effects can be selected a
priori or from effects plots
Degrees of freedom for t-tests and F-
tests should be adjusted accordingly
 Statistical tests for
replicated experiments
Error
estimated from negiglible terms
(Lack of Fit) is similar to MSPE

Source DF SS MS
Residual Error 28 300.38 10.73
Lack of Fit 4 45.38 11.34
Pure Error 24 255.00 10.63