Professional Documents
Culture Documents
ASSUMPTIONS OF REGRESSION
Curvilinear Relationship
Predicted Y'
Linear Relationship
Predicted Y'
Figure 1. Example of curvilinear and linear relationships with standardized residuals by standardized predicted values
r
2
*
12
r
rr
12
11
22
(1)
Table 1
Values of r and r2 after correction for attenuation
1.00
Observed r
.10
.20
.40
.60
r
.10
.20
.40
.60
.80
r2
.01
.04
.16
.36
r
.13
.25
.50
.75
r2
.02
.06
.25
.57
Reliability of DV and IV
.70
.60
r
r2
r
r2
.14
.02
.17
.03
.29
.08
.33
.11
.57
.33
.67
.45
.86
.74
---
.50
r
.20
.40
.80
--
r2
.04
.16
.64
--
Note: for simplicity we show an example where both IV and DV have identical reliability estimates. In some of these
hypothetical examples we would produce impossible values, and so do not report these.
r12*.3 =
r12*.3 =
(2)
(3)
Table 2
Values of r12.3 and r212.3 after correction low reliability
Reliability of Covariate
Examples:
.80
.70
.60
r12
r13
r23
Observed r12.3
r12.3
r12.3
r12.3
.3
.3
.3
.23
.21
.20
.18
.5
.5
.5
.33
.27
.22
.14
.7
.7
.7
.41
.23
.00
-.64
.7
.3
.3
.67
.66
.65
.64
.3
.5
.5
.07
-.02
-.09
-.20
.5
.1
.7
.61
.66
.74
.90
Note: In some examples we would produce impossible values that we do not report.
ASSUMPTIONS OF REGRESSION
Heteroscedasticity
Homoscedasticity
Predicted Y'
Heteroscedasticity
Predicted Y'
Predicted Y'
Conclusion
Homoscedasticity means that the variance of
errors is the same across all levels of the IV. When
the variance of errors differs at different values of the
IV, heteroscedasticity is indicated. According to
Berry and Feldman (1985) and Tabachnick and Fidell
(1996) slight heteroscedasticity has little effect on
significance tests; however, when heteroscedasticity
is marked it can lead to serious distortion of findings
and seriously weaken the analysis thus increasing the
possibility of a Type I error.
This assumption can be checked by visual
examination of a plot of the standardized residuals
(the errors) by the regression standardized predicted
value. Most modern statistical packages include this
as an option. Figure 3 show examples of plots that
might result from homoscedastic and heteroscedastic
data.
Ideally, residuals are randomly scattered around
0 (the horizontal line) providing a relatively even
distribution. Heteroscedasticity is indicated when the
residuals are not evenly scattered around the line.
There are many forms heteroscedasticity can take,
such as a bow-tie or fan shape. When the plot of
residuals appears to deviate substantially from
References
Berry, W. D., & Feldman, S. (1985). Multiple
Regression in Practice. Sage University Paper Series
on Quantitative Applications in the Social Sciences,
series no. 07-050). Newbury Park, CA: Sage.
Cohen, J., & Cohen, P. (1983). Applied multiple
regression/correlation analysis for the behavioral
sciences.
Hillsdale, NJ: Lawrence Erlbaum
Associates, Inc.
Nunnally, J. C. (1978). Psychometric Theory
(2nd ed.). New York: McGraw Hill.
Osborne, J. W. (2001). A new look at outliers
and fringeliers: Their effects on statistic accuracy
and Type I and Type II error rates. Unpublished
manuscript, Department of Educational Research and
Leadership and Counselor Education, North Carolina
State University.
Osborne, J. W., Christensen, W. R., & Gunter, J.
(April, 2001). Educational Psychology from a
Statisticians Perspective: A Review of the Power
and Goodness of Educational Psychology Research.
Paper presented at the national meeting of the
American Education Research Association (AERA),
Seattle, WA.
Pedhazur, E. J., (1997). Multiple Regression in
Behavioral Research (3rd ed.). Orlando, FL:Harcourt
Brace.
Tabachnick, B. G., Fidell, L. S. (1996). Using
Multivariate Statistics (3rd ed.). New York: Harper
Collins College Publishers
Tabachnick, B. G., Fidell, L. S. (2001). Using
Multivariate Statistics (4th ed.). Needham Heights,
MA: Allyn and Bacon.