Aims and Objectives ANALYSIS OF VARIANCE (ANOVA)

Wah Mong Weh Jabatan Matematik IPG KSAH Understand the basic principles of ANOVA -why it is done? - what it tells us? To conduct one-way independent ANOVA by hand

When And Why?

When do we use ANOVA?
When we would like to compare means from 3 or more groups.

AOVA
One-way analysis of variance (ANOVA) is a hypothesis testing technique that is used to compare means from three or more populations.

Why not use lots of t-tests?

Cant look at several independent variables at once Inflates the Type I error Need many t tests when the number of groups increase. [See Andy Field handout]

ANOVA
ANOVA adalah kaedah ujian hipotesis bagi mengenalpasti perbezaan min yang wujud dalam dua ataupun lebih sample ujian. Tujuan utama ANOVA adalah menentukan sama ada perbezaan sample disebabkan kesilapan proses sample ataupun kesan rawatan yang sistematik .
5

JENIS-JENIS ANOVA
a. One way ANOVA - yang melibatkan satu pembolehubah bebas. b. Two way ANOVA - melibatkan dua pembolehubah bebas.

ANOVA SATU HALA

Ujian statisti yan! di!una an untu mencari perbe"aan di antara ti!a atau lebih min #ipotesis nol ba!i ANOVA ialah $min-min populasi daripada tempat sampelsampel itu diambil adalah sama% H0: 1 = 2 = 3 = . . . k
7

ANOVA SATU HALA

#ipotesis alternatif ialah $ &e uran!- uran!nya terdapat satu pasan!an min populasi yan! berbe"a% H1: 1 2

Testing the Hypothesis

The null hypothesis that the three (or more) population means are equal. That is written as H0: 1 = 2 = 3 = . . . k H1: At least one mean is different

Theory of ANOVA
We calculate how much variability there is between scores. -Total sum of squares (SST ) We then calculate how much of this variability can be explained by the model we fit to the data -how much variability is due to the experimental manipulation, Model Sum of Squares (SSM)
9 10

Theory of ANOVA
How much cannot be explained How much is due to individual differences in performance, Residual Sum of Squares (SSR ) We compare the amount of variability explained by the model (experiment), to the error in the model ( individual differences)
- The ratio is called the F-ratio

Test Statistic for One-Way ANOVA

F=

variance between samples variance within samples

A excessively large F test statistic is evidence against equal population means. Thus, the null will be rejected.

11

12

 The variance is calculated in two different ways and the ratio of the two values is formed'

MSB, Mean Square Between, the variance between samples, measures the differences related to the treatment given to each sample. 2. MSW Mean Square Within, the variance within samples, measures the differences related to entries within the same sample. The variance within samples is due to sampling error.
13 14

MS B F= MSW

One-Way ANOVA Assumptions

1. 2. 3. 4. 5. The populations have normal distributions. The populations have the same variance 2(or standard deviation ). The samples are simple random samples. The samples are independent of each other. The different samples are from populations that are categorized in only one way.

ANOVA methods require the F-distribution

1. The F-distribution is not symmetric; it is skewed to the right. 2. The values of F can be 0 or positive, they cannot be negative. 3. There is a different F-distribution for each pair of degrees of freedom for the numerator and denominator. 4. Critical values of F can be found from Tables
15 16

Critical Value of F
Right-tailed test Degree of freedom with k samples of the same size n is given by:

F distribution: General Shape

Not symmetric (skewed to the right)

F [3,16]

numerator df

= k -1

nonnegative values only

denominator df = k(n -1) or N-k

Illustrative graph for 4 samples with 5 members each. k-1 = 4-1 = 3

17

N-k = 20-4= 16
18

Relationships Among Components of ANOVA

Taburan persampelan yan! di!una an ialah taburan (' & ala statisti yan! di!una an ialah nisbah (

19

20

EQUAL SAMPLE SIZES

Variance between samples = ns x
2

CALCULATING ANOVA
BY HAND

where s x = variance of sample means

Variance within samples = s p

2

Where s p = pooled variance (or the mean of the sample variances)

21 22

Calculations with Unequal Sample Sizes

ni(xi - x)
variance between samples F = variance = within samples
2

Calculations with Unequal Sample Sizes

ni(xi - x)2
variance between samples F = variance = within samples

k -1 (ni - 1)si (ni - 1)

2

k -1 (ni - 1)si2 (ni - 1)

See Formula Booklet pp. 20-21

where x = mean of all sample scores combined, also known as grand mean

23

24

Calculations with Unequal Sample Sizes

ni(xi - x)
variance between samples F = variance = within samples
2

Calculations with Unequal Sample Sizes

ni(xi - x)2
variance between samples F = variance = within samples

k -1 (ni - 1)si (ni - 1)

2

k -1 (ni - 1)si2 (ni - 1)

where x = mean of all sample scores combined

where x = mean of all sample scores combined

k = number of population means being compared

k = number of population means being compared ni = number of values in the ith sample

25

26

Calculations with Unequal Sample Sizes

ni(xi - x)2
variance between samples F = variance = within samples

Calculations with Unequal Sample Sizes

ni(xi - x)2
variance between samples F = variance = within samples

k -1 (ni - 1)si2 (ni - 1)

k -1 (ni - 1)si2 (ni - 1)

where x = mean of all sample scores combined

where x = mean of all sample scores combined

k = number of population means being compared ni = number of values in the ith sample xi = mean of the ith sample

k = number of population means being compared ni = number of values in the ith sample xi = mean of the ith sample si = variance of values in the ith sample
27 28 2

Key Components of ANOVA Method

SS(total), or total sum of squares, is a
measure of the total variation (around x) in all the sample data combined.

Key Components of ANOVA Method

SS(total), or total sum of squares, is a measure of the total variation (around x) in all the sample data combined and can be obtained using either formula below.

SS (Total ) = x

( x )
N
2

SS(total) = (x - x)
29

30

Key Components of ANOVA Method

SS(treatment) is a measure of the variation
between the samples [see Formula 6.119ii)]. In one-way ANOVA, SS(treatment) is sometimes referred to as SS(model). Because it is a measure of variability between the sample means, it is also referred to as SS (between groups) or SS (between samples).

Key Components of ANOVA Method

SS(treatment) is a measure of the variation between the samples. It is computed in the following way: SS(treatment) = n1(x1 - x)2 + n2(x2 - x)2 + . . . nk(xk - x)2 = ni(xi - x)2

31

32

Key Components of ANOVA Method

SS(error) is a sum of squares representing the variability
that is assumed to be common to all the populations being considered [Also called SS(residual)]
2 SS(error) = (n1 -1)s1 + (n2 -1)s2 2 + (n3 -1)s3 . . . nk(xk -1)si 2 2

Key Components of ANOVA Method

SS(total) = SS(treatment) + SS(error)

= (ni - 1)si

33

34

Mean Squares (MS)

Sum of Squares SS(treatment) and SS(error) divided by corresponding number of degrees of freedom give the mean squares.

MS (Treatment / Model / Group) MSTR / MSM / MSG is the mean square for
treatment or model or group and is obtained as follows:

MS (treatment) =

SS (treatment) k-1

35

36

MS (Error or Residual)
MS (error) is mean square for error, obtained as follows: MS (error) =
SS (error) N-k

Mean Squares (MS)

MS (error) is mean square for error
obtained as follows:
SS (error) N-k SS (total) N-1

MS (error) = MS (total) =

37

38

Test Statistic for ANOVA with Unequal Sample Sizes F=

MS (treatment) MS (error)

Example 1 (Equal sample sizes)

A researcher wishes to try three different techniques to lower the blood pressure of individuals diagnosed with high blood pressure. The subjects are randomly assigned to three groups; the first group takes medication, the second group exercises, and the third group follows a special diet. After four weeks, the reduction in each person's blood pressure is recorded. At =0.05, test the claim that there is no difference among the means. The data are shown
39 40

df = k -1 Denominator df = N - k
Numerator

Medicati on 10 12 9 15 13
Sample means sample sd sample var

Exercis e 6 8 3 0 2

Diet 5 9 12 8 4

41