Experimental Analysis

and Design

INTRODUCTION TO
ANALYSIS AND DESIGN
EXPERIMENT

PROF. GEVELYN B. ITAO

Master of Engineering (Material Science)
 Experimental Analysis and Design
Course Requirements
1. Major Exams 50%
2. Take Home Exams (online) 25%
3. Special Project(Oral and Written) 25%
Proposal Submission Last week
of March
Final Presentation 1st week of May
Passing 60%
Textbook: Design and Analysis of Experiments
by: Douglas Montgomery
Reference: Experimental Analysis by Lipson
 Experimental Analysis and Design
Special Project/ Research Study
I. Introduction
1. Statement of the Problem
2. Objectives
3. Hypothesis (Experimental)
4. Scope and Limitation
II. Methodology
1. Data Gathering
2. Statistical Design
III. Results and Discussion
1. Gathered Data
2. Statistical Analysis
3. Results (relate the statistical standing)
IV. Conclusions and Recommendation
Course and Action
 Experimental Analysis and Design

atistical Design of Experiments

- refers to the process of planning the
experiment so that appropriate data that
can be analyzed by statistical methods
can be collected, resulting in valid and
objective conclusions.
Everyone involved in the experiment must
have a clear idea in advance of exactly
what is to be studied, how the data is to be
collected, and at least a qualitative
understanding of how these data are to be
analyzed.
 Experimental Analysis and Design

idelines for Designing Experiments

1. Recognition of and Statement of the

Problem
2. Choice of Factors and Levels
3. Selection of the Response Variable
4. Choice of Experimental Design
5. Performing the Experiment
6. Data Analysis
7. Conclusions and Recommendations
 Experimental Analysis and Design

Terminologies

Experiment An investigation in which the

investigator applies some treatments to
experimental units and then observes the
effect of the treatments on the experimental
units by measuring one or more response
variables.

Treatment a condition or set of conditions

applied to experimental units in an
experiment.
 Experimental Analysis and Design

Terminologies

Experimental Unit - the physical entity to

which a treatment is randomly assigned and
independently applied.
Observational Unit - the unit on which a
response variable is measured. There is often
a one-to-one correspondence between
experimental units and observational units,
but that is not always true.
Response Variable a characteristic of an
experimental unit that is measured after
treatment and analyzed to assess the effects
 Experimental Analysis and Design
Example 1:
An experiment was conducted to study the effects of
three moisture levels on the strength in concrete
hollowblocks.
A total of 36 concrete hollowblocks were made in 12
molds with 3 concrete hollowblocks in each mold.
The 3 moisture levels (low, medium, and high) were
randomly assigned to the 12 molds with 4 molds for
each moisture level.
After three weeks, sample was collected from each
concrete hollowblocks.
Each of the 36 samples was tested with the measure
strength.
 Experimental Analysis and Design
Example 1 (Contd.):

Low Low Low Low

Mediu Mediu Mediu Mediu

m m m m

Hig Hig Hig Hig

h h h h
 Experimental Analysis and Design
Example 1 (Contd.):

1. Name the treatments in this experiment.

2. Name the experimental units in this experiment.

3. Name the observational units in this experiment.

4. Name the response variable or variables in this

experiment.
 Experimental Analysis and Design
Example 1 (Contd.):

1. Each of the three moisture levels represents a

treatment.
2. The moisture levels were randomly assigned to
the molds, so the molds are the experimental
units. A mold consisting of 3 hollowblocks is one
experimental unit.
3. Strength was measured for each hollowblocks, so
the hollowblocks are the observational units.
4. Each sample provides one response variable.
Thus, we will have several thousand response
variables in this example.
 Experimental Analysis and Design

Terminologies (Contd.)

Explanatory Variable a variable that can

potentially be used to explain variation in a
response variable.

Factor an explanatory variable that can take

any one of two or more values.

Levels the different values of a factor.

Treatment Factor a factor whose levels are

chosen and controlled by the researcher to
understand how one or more response
 Experimental Analysis and Design

Terminologies (Contd.)

Treatment Design the collection of

treatments used in an experiment.

Full Factorial Treatment Design

treatment design in which the treatments
consist of all possible combinations involving
one level from each of the treatment factors.
 Experimental Analysis and Design
Example 2:
An experiment was conducted to gauge the effects of
water hyacinth addition and drying on compressive
strength in concrete material.
A total of 40 concrete materials were prepared. Half of
the 40 concrete materials were prepared and fired in
kiln. The other 20 concrete materials were prepared
by normal drying.
Within each firing/drying group, four water hyacinth
addition (0, 10, 20, and 40% by weight) were
conducted to concrete materials with 5 concrete
materials per firing/drying.
At the conclusion of the study, compressive strength
was measured for each concrete material.
 Experimental Analysis and Design
Example 2 (Contd.):

1. Name the treatment factors used in this

experiment.

2. Name the levels of each factor.

3. Name the treatments used in this experiment.

4. Was a full factorial treatment design used?

5. Name the experimental units used in this

experiment.

6. Name the observational units used in this

experiment.
 Experimental Analysis and Design
Example 2 (Contd.):

1. The treatment factors are drying and water

hyacinth addition.
2. The levels of the factor drying are firing and
normally dried. The levels of the factor water
hyacinth are 0, 10, 20, and 40 % by weight.
3. Each combination of drying and water hyacinth is
one treatment. (F0, F10, F20, F40, N0, N10, N20,
N40).
4. A full-factorial treatment design was used because
all possible combinations of drying and water
hyacinth addition were considered.
5. Each hollowblock is an experimental unit and also
 Experimental Analysis and Design

Terminologies

Completely Randomized Design (CRD)

experimental design in which, for given
number of experiment units per treatment, all
possible assignments of treatments to
experimental units are equally likely.
Block a group of experimental units that,
prior to treatment, are expected to be more
like one another (with respect to one or more
response variables) than experimental units in
general.
 Experimental Analysis and Design

Terminologies (Contd.)

Randomized Complete Block Design (RCBD)

experimental design in which separate and
completely randomized treatment
assignments are made for each of multiple
blocks in such a way that all treatments have
at least one experimental unit in each block.
 Experimental Analysis and Design
e Fundamental Experimental Design Concepts
buted to R.A. Fisher
Randomization random assignment of
treatments to experimental units.

Blocking grouping similar experimental units

together and assigning different treatments
within such groups of experimental units.

Replication applying a treatment

independently to two or more experimental
units.
 Experimental Analysis and Design
Example 3:
Suppose an experiment is to be conducted to study
the effects on tensile strength of 5 treatments (A, B,
C, D, and E) producing different steel types.

A total of 25 testing machines and a total of 25 steel

types, located on 5 companies with 5 steel types on
each company, are available for the experiment.

Which of the following designs is better from a

statistical standpoint?
 Experimental Analysis and Design
Example 3 (Contd.):
Design 1: To reduce variability within treatment
groups, randomly assign the 5 treatments to the 5
companies so that all 5 steel types on any one
company receive the same treatment. Measure
tensile strength using one testing machine for each
steel type.

Design 2: Randomly assign the 5 treatments to the 5

steel types within each company so that all 5
treatments are represented on each company.
Measure tensile strength using one testing machine
for each steel type.
 Experimental Analysis and Design
Example 3 (Contd.):
Design 1 Design 2

Company 1: B B B B B Company 1: A B E D C

Company 2: D D D D D Company 2: E D A C B

Company 3: A A A A A Company 3: C D E A B

Company 4: E E E E E Company 4: A B E
CD

Company 5: C C C C C Company 5: C A D B E
 Experimental Analysis and Design
Example 3 (Contd.):

1. Name the observational units in each design.

2. Name the experimental units in each design.

3. Is blocking used for either design? If so, describe

the blocks.

4. Describe the level of replication for each

experimental design.
 Experimental Analysis and Design
Example 3 (Contd.):
1. Steel types are the observational units in both
designs.

2. Companies are the experimental units in Design

1, and steel types are the experimental units in
Design 2.

3. Design 2 is a randomized complete block design

(RCBD) with a group of 5 steel types on a
company serving as a block of experimental units.

4. Design 1 has no replication because there is only

1 experimental unit for each treatment. Design 2
has 5 replications per treatment.
 Experimental Analysis and Design
Example 3 (Contd.):
Design 2 is by far the better design.

We can compare treatments directly among steel

typess that share the same company.

With Design 1 it is impossible to separate differences

in tensile strength due to treatment effects from
differences in tensile strength that might be due to
company effects.
 Experimental Analysis and Design

SIMPLE COMPARATIVE EXPERIMENTS

- experiments performed to
compare two conditions
(sometimes called treatments)

Statistical Hypothesis

An assertion or conjecture concerning one or

more populations
 Experimental Analysis and Design
pothesis Testing for Means of Normal
Distributions:
Null Hypothesis, H0
Any hypothesis we wish to test
It will always be stated so as to specify an
exact value of the parameter
Any rejection of a null hypothesis leads to
the acceptance of the alternative
hypothesis
Alternative Hypothesis, H1
Unlike the null hypothesis, it allows for the
possibility of several values
 Experimental Analysis and Design
pothesis Testing for Means of Normal
Distributions:
Possible Situations for Testing a
Statistical HYPOTHESIS:

DECISION H0 is TRUE H1 is FALSE

Accept, H0 Correct
Type II error
decision
Reject, H0 Type I error Correct decision
 Experimental Analysis and Design
pothesis Testing for Means of Normal
Distributions:
- Level of Significance of the Test
the probability of committing a Type I
error
also called as the size of critical
region
1- - Level of Confidence of the Test
maximum risk of committing a Type I

error
the probability of committing a Type
II
1-error
- Quality/Power of Test
Experimental Analysis and Design
Steps in Testing Hypothesis: Classical
Approach
1. State the null hypothesis H0 that = 0
2. Choose an appropriate alternative
hypothesis H1 from one of the alternatives
3. that <a 0significance
Choose , > 0, or level
0 of size .

4. Select the appropriate test statistic and

establish the critical region based on .
5. Compute the value of the test statistic from
the sample data.
Experimental Analysis and Design
Steps in Testing Hypothesis: Classical
Approach
6. Decision: Reject H0 if the test statistic has a
value in the critical region (or if the
computed P-value is less than or equal to the
desired significance level ); otherwise, do
not reject H0.
 Experimental Analysis and Design

P-value Approach

ue - is the lowest level (of significance) at

the observed value of the test statistic is sign
Criteria:
P : reject H0
P > : accept H0
Experimental Analysis and Design
Significance Testing (P-value Approach):
1. State the null hypothesis H0 that = 0
2. Choose an appropriate alternative
hypothesis H1 from one of the alternatives
3. that < 0P-value
Compute , > 0,based
or on
0 computed value

of test statistic.
4. Use judgment based on P-value and
knowledge of scientific system.
5. Decision: Reject H0 if the computed P-value
is less than or equal to the desired
significance level ; otherwise, do not reject
H .
Experimental Analysis and Design
One- and Two-Tailed Tests
One-tailed test:
A test of any statistical hypothesis, where
the alternative is one-sided, such as

or

The rejection The rejection

region falls region falls
on the right tail of on the left tail of
the the
associated associated
Experimental Analysis and Design
One- and Two-Tailed Tests
Two-tailed test:
A test of any statistical hypothesis, where
the alternative is two-sided, such as

The rejection region is split into two parts,

often having equal probabilities placed in
each tail of the distribution.
Experimental Analysis and Design
Guidelines in Stating H0 and H1:
Null hypothesis is always stated with equality
Null hypothesis includes any value not stated
in the alternative hypothesis
1. Claims of < or > types
Follow the directed sign as stated

or

If claim is greater If claim is less

than, superior to, than, inferior to,
and so on. and so on.
Experimental Analysis and Design
Guidelines in Stating H0 and H1:
Null hypothesis is always stated with equality
Null hypothesis includes any value not stated
in the alternative hypothesis
2. Claims of or types
Follow the opposite of the directed sign

or

If claim is at least, If claim is equal to

no more than, or greater than, at
and so on. most, and so on.
Experimental Analysis and Design
Guidelines in Stating H0 and H1:
Null hypothesis is always stated with equality
Null hypothesis includes any value not stated
in the alternative hypothesis
3. Claims with no indicated directions

Two-sided tail
Critical region splits into two equal parts
Experimental Analysis and Design
One- and Two-Sample Tests of Hypotheses
Illustration
A certain type of vaccine is known to be
only 25% effective after a period of 2
years. To determine if a new and somewhat
more expensive vaccine is superior in
providing protection against the same
virus for a longer period of time, a random
sample of 20 people is inoculated by the
new vaccine. If more than 8 of those
receiving the new vaccine surpass the 2-
year period without contracting the virus,
the new vaccine will be considered
superior to the one presently in use.
Experimental Analysis and Design
One- and Two-Sample Tests of Hypotheses
Illustration
H0: p = 0.25
The new vaccine is no better than the old
vaccine
H1: p > 0.25
The new vaccine is better than the old
vaccine
All possible scores greater than 8 (X > 8 )
constitute the critical region
All possible scores less than or equal to 8
(X 8 ) constitute the acceptance region
Experimental Analysis and Design
One- and Two-Sample Tests of Hypotheses
Types of Error in the Conclusion
Type I error, :
Rejection of the null hypothesis when it is
true
Illustration: Suppose that the new vaccine
is no better than the old vaccine.

After testing, more than 8 surpass the 2-

year period without contracting the
disease. Hence, this will lead to the
rejection of the null hypothesis and accept
the alternative hypothesis that the new
vaccine is better than the old vaccine.
Experimental Analysis and Design
One- and Two-Sample Tests of Hypotheses
Types of Error in the Conclusion
Type II error, :
Acceptance of the null hypothesis when it
is false
Illustration: Suppose that the new vaccine
is better than the old vaccine.

After testing, less than 8 surpass the 2-

year period without contracting the
disease. Hence, this will lead to the
acceptance of the null hypothesis and
reject the alternative hypothesis that the
new vaccine is better than the old vaccine.
 Experimental Analysis and Design
One- and Two-Sample Tests of Hypotheses
Illustration:
A Type I error () will occur when more than 8
individuals surpass the 2-year period without
contracting the virus using a new vaccine that
is actually equivalent to the one in use.
= P(type I error) = P(X > 8 when p = 1/4) =

A critical value of 0.0409 is very small and

therefore it is unlikely to commit a type I
error.
 Experimental Analysis and Design
One- and Two-Sample Tests of Hypotheses
Illustration:
A Type II error () will occur when less than 8
individuals surpass the 2-year period without
contracting the virus using a new vaccine that
is actually better than the one in use.

H1: p 1/2

= P(type II error) = P(X 8 when p = 1/2)

Experimental Analysis and Design
One- and Two-Sample Tests of Hypotheses
For a fixed sample size, a decrease in the
probability of one error will usually result in
an increase in the probability of the other
error.
But the probability of committing both types
of error can be reduced by increasing the
sample size.
Experimental Analysis and Design
One- and Two-Sample Tests of Hypotheses
Consider the same problem but this time the
number of samples is increased from 20 to
100 individuals.
If more than 36 individuals surpass the 2-
year period, reject the null hypothesis that p
= and accept the alternative hypothesis
that p > .
 Experimental Analysis and Design
One- and Two-Sample Tests of Hypotheses
Consider the same problem but this time the
number of samples is increased from 20 to
100 individuals.
If more than 36 individuals surpass the 2-
year period, reject the null hypothesis that p
= and accept the alternative hypothesis
that p > .
= P(type I error) = P(X > 36 when p = 1/4) =
P(Z > 2.66)
Experimental Analysis and Design
One- and Two-Sample Tests of Hypotheses
Consider the same problem but this time the
number of samples is increased from 20 to
100 individuals.
If less than 36 individuals surpass the 2-year
period, accept the null hypothesis that p =
and reject the alternative hypothesis that p >
, say p = 1/2.
Experimental Analysis and Design
One- and Two-Sample Tests of Hypotheses
Consider the same problem but this time the
number of samples is increased from 20 to
100 individuals.
If less than 37 individuals surpass the 2-year
period, accept the null hypothesis that p =
and reject the alternative hypothesis that p >
, say p = 1/2.
= P(type I error) = P(X < 37 when p = 1/2) =
P(Z < 2.7)
Experimental Analysis and Design
Advance Reading:
Hypothesis testing for single mean
Hypothesis testing for difference of two
means
Hypothesis testing for paired observation