Business Statistics, 5

th
ed.
by Ken Black



Chapter 15

Multiple Regression
Analysis
Discrete Distributions
PowerPoint presentations prepared by Lloyd Jaisingh,
Morehead State University

Learning Objectives
Develop a multiple regression model.
Understand and apply significance tests of the regression
model and its coefficients.
Compute and interpret residuals, the standard error of the
estimate, and the coefficient of determination.
Interpret multiple regression computer output.
Regression Models

Probabilistic Multiple Regression Model

Y = |
0
+ |
1
X
1
+ |
2
X
2
+ |
3
X
3
+ . . . + |
k
X
k
+ c

Y = the value of the dependent (response) variable
|
0
= the regression constant
|
1
= the partial regression coefficient of independent variable 1
|
2
= the partial regression coefficient of independent variable 2
|
k
= the partial regression coefficient of independent variable k
k = the number of independent variables
c = the error of prediction
Estimated Regression Model
s t variable independen of number =
t coefficien regression of estimate
3 t coefficien regression of estimate
2 t coefficien regression of estimate
1 t coefficien regression of estimate
constant regression of estimate
of value predicted

3
2
1
0
3 3 2 2 1 1 0
k
k
Y Y where
Y
b
b
b
b
b
X b X b X b X b b
k
k k
=
=
=
=
=
=
+ + + + + =
Multiple Regression Model with Two
Independent Variables (First-Order)
Y
where
Y
where Y
X X
b b X b X
b
b
b
= + + +
=
=
= + +
=
=
=
=
0 1
1
2
2
0
1
2
0 1 1 2 2
0
1
2
| | |
|
|
|
c
c
:

:

= the regression constant
the partial regression coefficient for independent variable 1
the partial regression coefficient for independent variable 2
= the error of prediction
predicted value of Y
estimate of regression constant
estimate of regression coefficient 1
estimate of regression coefficient 2
Population
Model
Estimated
Model
Response Plane for First-Order Two-
Predictor Multiple Regression Model
X
1
X
2
Response Plane
Y
1
Vertical Intercept
Y
c
Least Squares Equations for k = 2
0 1 1 2 2
0 1 1 1
2
2 1 2 1
0 2 1 1 2 2 2
2
2
b b X b X Y
b X b X b X X X Y
b X b X X b X X Y
n + + =
+ + =
+ + =



Real Estate Data
Observation Y X
1
X
2
Observation Y X
1
X
2
1 63.0
65.1
1,605 35 13 79.7 2,121 14
2 2,489 45 14 84.5 2,485 9
3 69.9
7
1,553 20 15 96.0 2,300 19
4 76.8 2,404 32 16 109.5 2,714 4
5 73.9 1,884 25 17 102.5 2,463 5
6 77.9 1,558 14 18 121.0 3,076 7
7 74.9 1,748 8 19 104.9 3,048 3
8 78.0 3,105 10 20 128.0 3,267 6
9 79.0 1,682 28 21 129.0 3,069 10
10 63.4 2,470 30 22 117.9 4,765 11
11 79.5 1,820 2 23 140.0 4,540 8
12 83.9 2,143 6
Market
Price
($1,000)
Square
Feet
Age
(Years)
Market
Price
($1,000)
Square
Feet
Age
(Years)
Real Estate Data
5000
4000
Price
60
90
120
3000
150
SquareFeet
0
2000 15
30
45 Age
Real Estate Data
MINITAB Output
for the Real Estate Example
The regression equation is
Price = 57.4 + 0.0177 Sq.Feet - 0.666 Age

Predictor Coef StDev T P
Constant 57.35 10.01 5.73 0.000
Sq.Feet 0.017718 0.003146 5.63 0.000
Age -0.6663 0.2280 -2.92 0.008

S = 11.96 R-Sq = 74.1% R-Sq(adj) = 71.5%
Analysis of Variance

Source DF SS MS F P
Regression 2 8189.7 4094.9 28.63 0.000
Residual Error 20 2861.0 143.1
Total 22 11050.7
Predicting the Price of Home
( ) ( )
dollars thousand 658 . 93
12 666 . 0 2500 0177 . 0 4 . 57

, 12 and 2500
666 . 0 0177 . 0 4 . 57

2 1
2 1
=
+ =
= =
+ =
Y
X X
For
X X
Y
Evaluating the Multiple
Regression Model
H
H
k
a
0
1 2 3
0 :
:
| | | | = = = = =
=

At least one of the regression coefficients is 0

H
H
H
H
H
H
H
H
a a
a
k
a
k
0
1
1
0
3
3
0
2
2
0
0
0
0
0
0
0
0
0
:
:
:
:
:
:
:
:
|
|
|
|
|
|
|
|
=
=
=
=
=
=
=
=

Significance
Tests for
Individual
Regression
Coefficients
Testing
the
Overall
Model
Testing the Overall Model for the
Real Estate Example
0 is ts coefficien regression the of one least At :
0 :
2
1
0
=
=
=
a H
H
| |
MSR
SSR
k
MSE
SSE
n k
F
MSR
MSE
= =

=
1
ANOVA
df
SS MS F p
Regression 2 8189.723 4094.86 28.63 .000
Residual (Error) 20 2861.017 143.1
Total 22 11050.74
. , ,
.
. . ,
01 2 20
585
28 63 585
F
F
Cal
=
= > reject H . 0
Significance Test
of the Regression
Coefficients for
the Real Estate
Example
H
H
H
H
a
a
0
1
1
0
2
2
0
0
0
0
:
:
:
:
|
|
|
|
=
=
=
=
t
Cal
= 5.63 > 2.086, reject H
0
.
Coefficients Std Dev t Stat p
x
1
(Sq.Feet) 0.0177 0.003146 5.63 .000
x
2
(Age) -0.666 0.2280

-2.92 .008
t
.025,20
= 2.086
Residuals and Sum of Squares Error
for the Real Estate Example
SSE
Observation Y Observation Y
1 43.0 42.466 0.534 0.285 13 59.7 65.602 -5.902 34.832
2 45.1 51.465 -6.365 40.517 14 64.5 75.383 -10.883 118.438
3 49.9 51.540 -1.640 2.689 15 76.0 65.442 10.558 111.479
4 56.8 58.622 -1.822 3.319 16 89.5 82.772 6.728 45.265
5 53.9 54.073 -0.173 0.030 17 82.5 77.659 4.841 23.440
6 57.9 55.627 2.273 5.168 18 101.0 87.187 13.813 190.799
7 54.9 62.991 -8.091 65.466 19 84.9 89.356 -4.456 19.858
8 58.0 85.702 -27.702 767.388 20 108.0 91.237 16.763 280.982
9 59.0 48.495 10.505 110.360 21 109.0 85.064 23.936 572.936
10 63.4 61.124 2.276 5.181 22 97.9 114.447 -16.547 273.815
11 59.5 68.265 -8.765 76.823 23 120.0 112.460 7.540 56.854
12 63.9 71.322 -7.422 55.092 2861.017

Y Y Y

( )
2
Y Y

Y Y Y

( )
2
Y Y

MINITAB Residual Diagnostics for
the Real Estate Problem
Residual
P
e
r
c
e
n
t
30 15 0 -15 -30
99
90
50
10
1
Fitted Value
R
e
s
i
d
u
a
l
140 120 100 80 60
30
15
0
-15
-30
Residual
F
r
e
q
u
e
n
c
y
24 12 0 -12 -24
6.0
4.5
3.0
1.5
0.0
Observation Order
R
e
s
i
d
u
a
l
22 20 18 16 14 12 10 8 6 4 2
30
15
0
-15
-30
Normal Probability Plot of the Residuals Residuals Versus the Fitted Values
Histogram of the Residuals Residuals Versus the Order of the Data
Residual Plots for Price
SSE and Standard Error
of the Estimate
e
S
SSE
n k
where
=

=

=
1
2861
23 2 1
1196 .
: n = number of observations
k = number of independent variables
SSE
ANOVA
df
SS MS F P

Regression 2 8189.7 4094.9 28.63 .000
Residual (Error) 20 2861.0 143.1
Total 22 11050.7
Coefficient of Multiple Determination (R
2
)
2
2
8189 723
11050 74
741
1 1
2861017
11050 74
741
R
R
SSR
SSY
SSE
SSY
= = =
= = =
.
.
.
.
.
.
SSE
ANOVA
df
SS MS F p
Regression 2 8189.7 4094.89 28.63 .000
Residual (Error) 20 2861.0 143.1
Total 22 11050.7
SS
YY

SSR
Adjusted R
2
adj
SSE
n k
SSY
n
R
.
.
.
. .
2
1
1
1
1
2861017
23 2 1
1105074
23 1
1 285 715 =

= =
ANOVA
df
SS MS F p
Regression 2 8189.7 4094.9 28.63 .000
Residual (Error) 20 2861.0 143.1
Total 22 11050.7
SS
YY

SSE
n-k-1
n-1
Demonstration Problem 15.1: Freight
Data
Country
Freight Cargo Shipped by
Road
(Million Short-Ton
Miles)
Length 0f Roads
(Miles)
Number of Commercial
Vehicles
China 278,806 673,239 5,010,000
Brazil 178,359 1,031,693 1,371,127
India 144,000 1,342,000 1,980,000
Germany 138,975 395,367 2,923,000
Italy 125,171 188,597 2,745,500
Spain 105,824 206,271 2,859,438
Mexico 96,049 157,036 3,758,034
Demonstration Problem 15.1: Excel
Output
Regression Statistics
Multiple R 0.812
R Square 0.659
Adjusted R Square 0.488
Standard Error 44273.86677
Observations 7
ANOVA
df SS MS F Sig. F
Regression 2 15148592381 7.57E+09 3.86 0.116
Residual 4 7840701114 1.96E+09
Total 6 22989293495
Coefficients Standard Error t Stat P-value
Intercept -26425.45085 67624.93769 -0.39 0.716
Length 0f Roads 0.101820862 0.043495015 2.34 0.079
Commercial Vehicles 0.04094856 0.017121018 2.39 0.075
Demonstration Problem 15.1: MINITAB
Output
MINITAB Residual Diagnostics for
Demonstration Problem 15.1
Residual
P
e
r
c
e
n
t
100000 50000 0 -50000 -100000
99
90
50
10
1
Fitted Value
R
e
s
i
d
u
a
l
250000 200000 150000 100000
50000
25000
0
-25000
-50000
Residual
F
r
e
q
u
e
n
c
y
50000 25000 0 -25000 -50000
2.0
1.5
1.0
0.5
0.0
Observation Order
R
e
s
i
d
u
a
l
7 6 5 4 3 2 1
50000
25000
0
-25000
-50000
Normal Probability Plot of the Residuals Residuals Versus the Fitted Values
Histogram of the Residuals Residuals Versus the Order of the Data
Residual Plots for Freight
Copyright 2008 John Wiley & Sons, Inc.
All rights reserved. Reproduction or translation
of this work beyond that permitted in section 117
of the 1976 United States Copyright Act without
express permission of the copyright owner is
unlawful. Request for further information should
be addressed to the Permissions Department, John
Wiley & Sons, Inc. The purchaser may make
back-up copies for his/her own use only and not
for distribution or resale. The Publisher assumes
no responsibility for errors, omissions, or damages
caused by the use of these programs or from the
use of the information herein.