15 views

Uploaded by Urban Yaron Todorovic

Matrix by Lucas

- IJET-V4I3P93.pdf
- MuhammadNaeemSarwar 510 Module3 29122011
- [IJCST-V4I1P12]:Vivek Solavande
- Week 1 - Worksheet
- Crt
- MODUL 1 MATEMATIK SPM
- HW1
- Python Primer
- A STUDY ON THE PROBLEMS FACED BY RURAL TAMIL MEDIUM STUDENTS IN PROFESSIONAL ENGINEERING COLLEGE USING CETD MATRIX
- MATHEMATIC Notes Form 5 All Topics
- NumPy _version1
- Answerkey01novCHSLmorning Www.qmaths.in
- Rtiwari Rd Book 12
- CS _GATE-2018_Paper_4-FEB-2018 Morning Session (1)
- Chapter 1098
- Journal FEA
- Matlab Getting Started
- Matrices and Row Operations
- The network of global corporate control
- Center of Mass Specifications

You are on page 1of 51

in matrix terms

in matrix terms

Yi 0 1 x i i

for i = 1,, n

Y1 0 1 x1 1

Y2 0 1 x 2 2

Yn 0 1 x n n

in matrix notation

Y1

Y

2

Yn

1 x1

1

1 x

2

2

0

1 xn

n

Y X

Definition of a matrix

An rc matrix is a rectangular array of symbols or

numbers arranged in r rows and c columns.

A matrix is almost always denoted by a single capital

letter in boldface type.

1 2

A

6

3

1 80 3.4

1 92 3.1

B 1 65 2.5

1 71 2.8

1 40 1.9

x12

x 22

x32

1 x 41

1 x51

1 x61

x 42

x52

x62

1 x11

1 x

21

1 x31

A column vector is an r1 matrix, that is, a matrix

with only one column.

2

q 5

8

A row vector is an 1c matrix, that is, a matrix with

only one row.

h 21 46 32 90

ordinary number, such as 29 or 2.

Matrix multiplication

Y X

The X in the regression function is an example of

matrix multiplication.

Two matrices can be multiplied together:

Only if the number of columns of the first matrix equals

the number of rows of the second matrix.

The number of rows of the resulting matrix equals the

number of rows of the first matrix.

The number of columns of the resulting matrix equals

the number of columns of the second matrix.

Matrix multiplication

If A is a 23 matrix and B is a 35 matrix

then matrix multiplication AB is possible.

The resulting matrix C = AB has rows

and columns.

Is the matrix multiplication BA possible?

If X is an np matrix and is a p1 column

vector, then X is

Matrix multiplication

3 2 1 5

1 9 7

90 101 106 88

C AB

5

4

7

3

41 38 27 59

8

1

2

6 9 6 8

The entry in the ith row and jth column of C is the inner

product (element-by-element products added together)

of the ith row of A with the jth column of B.

c11 1(3) 9(5) 7(6) 90

c24

The X multiplication in

simple linear regression setting

1 x1

1 x

2

0

1 xn

Matrix addition

Y X

The X+ in the regression function is an example

of matrix addition.

Simply add the corresponding elements of the two

matrices.

For example, add the entry in the first row, first column

of the first matrix with the entry in the first row, first

column of the second matrix, and so on.

have the same number of rows and columns.

Matrix addition

2 4 1 7 5 2 9 9 1

C A B 1 8 7 9 3 1 10 5 8

3 5 6 2 1 8 5 6 14

For example:

c11 2 7 9

c12 4 5 9

c23

simple linear regression setting

Y1

0 1 x1 1

Y

x

2

0

1 2

2

Y

X

Yn

0 1 x n n

in matrix notation

Y1

Y

2

Yn

1 x11

1 x

21

1 x n1

x12

x 22

xn2

x13

x 23

xn3

Y X

1

2

n

of the parameters

The p1 vector containing the estimates of the p

parameters can be shown to equal:

b0

b

1

b p 1

X X X Y

1

X' is the transpose of the X matrix.

Definition of

the transpose of a matrix

The transpose of a matrix A is a matrix, denoted

A' or AT, whose rows are the columns of A and

whose columns are the rows of A all in the same

original order.

1 5

A 4 8

7 9

A A

simple linear regression setting

1

X X

x1

1

x2

1 x1

1 1 x2

xn

1 xn

The (square) nn identity matrix, denoted In, is a

matrix with 1s on the diagonal and 0s elsewhere.

1 0

I2

0 1

The identity matrix plays the same role as the

number 1 in ordinary arithmetic.

9 7 1 0

4 6 0 1

The inverse A-1 of a square (!!) matrix A is the

unique matrix such that

A A I AA

Find X'X.

in simple linear regression setting

xi

soap

4.0

4.5

5.0

5.5

6.0

6.5

7.0

--38.5

yi

x i yi

suds so*su

33 132.0

42 189.0

45 225.0

51 280.5

53 318.0

61 396.5

62 434.0

--- ----347 1975.0

x i2

soap2

16.00

20.25

25.00

30.25

36.00

42.25

49.00

----218.75

b0

1

b X X X Y ?

b1

1

X X

x1

1 1

x 2 x n

1 x1

1 x

2

1 xn

in simple linear regression setting

Its very messy to determine inverses by hand. We let

computers find inverses for us.

X X X X

38.5

7

38

.

5

218

.

75

4.4643 0.78571

0.78571 0.14286

Therefore:

X X

38.5

7

38

.

5

218

.

75

4.4643 0.78571

0

.

78571

0

.

14286

Find X'Y.

in simple linear regression setting

xi

soap

4.0

4.5

5.0

5.5

6.0

6.5

7.0

--38.5

yi

x i yi

suds so*su

33 132.0

42 189.0

45 225.0

51 280.5

53 318.0

61 396.5

62 434.0

--- ----347 1975.0

x i2

soap2

16.00

20.25

25.00

30.25

36.00

42.25

49.00

----218.75

b0

1

b X X X Y ?

b1

1

X Y

x1

1 1

x 2 x n

y1

y

2

yn

linear regression setting

b X X

4.4643 0.78571

X Y

0.78571 0.14286

347

1975

b

0

.

78571

(

347

)

0

.

14286

(

1975

)

9

.

51

1

The regression equation is

suds = - 2.68 + 9.50 soap

Linear dependence

The columns of the matrix:

1 2 4 1

A 2 1 8 6

3 6 12 3

columns can be written as a linear combination of

another.

If none of the columns can be written as a linear

combination of another, then we say the columns are

linearly independent.

Linear dependence

is not always obvious

1 4 1

A 2 3 1

3 2 1

Formally, the columns a1, a2, , an of an nn matrix are

linearly dependent if there are constants c1, c2, , cn,

not all 0, such that:

c1a1 c2 a 2 cn a n 0

on regression

The inverse of a square matrix exists only if

the columns are linearly independent.

Since the regression estimate b depends on

(X'X)-1, the parameter estimates b0, b1, ,

cannot be (uniquely) determined if some of

the columns of X are linearly dependent.

about linear dependence

If the columns of the X matrix (that is, if

two or more of your predictor variables) are

linearly dependent (or nearly so), you will

run into trouble when trying to estimate the

regression function.

on regression

soap1

4.0

4.5

5.0

5.5

6.0

6.5

7.0

soap2 suds

8

33

9

42

10

45

11

51

12

53

13

61

14

62

* soap2 has been removed from the equation

The regression equation is

suds = - 2.68 + 9.50 soap1

Fitted values

y1 b0 b1 x1

y b b x

2

0

1 2

y n b0 b1 x n

Fitted values

The vector of fitted values

y Xb X X X X y

1

hat matrix H

H X X X X

1

That is:

X X X

y

X y Hy

e i yi y i

e1 y1 y1

e2 y2 y 2

en yn y n

for i = 1,, n

e1

e

2

en

y1 y1

y y

2

2

yn y n

function of the hat matrix

e1

e

2

y1 y1

y y

2

2

en

yn y n

Sum of squares

and the analysis of variance table

in matrix terms

Source

Regression

DF

SS

MS

p-1

SSR b X Y Y JY

n

SSR

p 1

MSR

MSE

Error

n-p

SSE Y Y bX Y

Total

n-1

SSTO Y Y

1

Y JY

n

SSE

n p

Sum of squares

In general, if you pre-multiply a vector by its transpose,

you get a sum of squares.

yy y1

y2

y1

y

n

2

yn y12 y22 yn2 yi2

i 1

yn

n

SSE yi y i

i 1

SSE y y y y

'

Previously, wed write:

SSTO y i y yi

2

1

SSTO Y Y Y JY

n

where J is a (square) nn matrix containing all 1s.

An example of

total sum of squares

If n = 2:

i 1

Yi

2

Y JY Y1

1 1 Y1

2

2

Y2

2

Y

Y

Y

1

1 2

2

Y

1

1

in matrix terms

Source

Regression

DF

SS

MS

p-1

SSR b X Y Y JY

n

SSR

p 1

MSR

MSE

Error

n-p

SSE Y Y bX Y

Total

n-1

SSTO Y Y

1

Y JY

n

SSE

n p

Model assumptions

As always, the error terms i are:

independent

normally distributed (with mean 0)

with equal variances 2

matrices and vectors?

The n1 random error term vector, denoted as ,

is:

1

2

random error term vector

The n1 mean error term vector, denoted as E(),

is:

E 1 0

E 0

2

E

0

E n 0

Definition

Assumption

Definition

random error term vector

The nn variance matrix, denoted as 2(), is

defined as:

1 2 1 ( 1 , 2 ) ( 1 , n )

2

2 ( 1 , 2 ) 2

( 2 , n )

2

2

2

n ( 1 , n ) ( 2 , n ) n

Off-diagonal elements are covariances between errors.

random error term vector

BUT, we assume error terms are independent

(covariances are 0), and have equal variances (2).

1 2 0 0

2

2 0

0

2

n 0

0

Just multiply each element of the matrix by the scalar.

For example:

1 4 0 2 8 0

2 7 6 5 14 12 10

1 3 2 2 6 4

random error term vector

2 0 0

2

0

0

2

2

0

0

Putting the regression function and assumptions all

together, we get:

Y X

where:

Y is a (

is a (

X is an (

is an (

) vector of independent, normal error

terms with mean 0 and (equal) variance 2I.

- IJET-V4I3P93.pdfUploaded byInternational Journal of Engineering and Techniques
- MuhammadNaeemSarwar 510 Module3 29122011Uploaded bynsnsnsns
- [IJCST-V4I1P12]:Vivek SolavandeUploaded byEighthSenseGroup
- Week 1 - WorksheetUploaded byAngel Manaloto
- CrtUploaded bykbmn2
- MODUL 1 MATEMATIK SPMUploaded byRoslan Abdullah
- HW1Uploaded byStacy Tash
- Python PrimerUploaded byAakash Kardam
- A STUDY ON THE PROBLEMS FACED BY RURAL TAMIL MEDIUM STUDENTS IN PROFESSIONAL ENGINEERING COLLEGE USING CETD MATRIXUploaded byIntegrated Intelligent Research
- MATHEMATIC Notes Form 5 All TopicsUploaded byKian Ying
- NumPy _version1Uploaded byAsim Riaz
- Answerkey01novCHSLmorning Www.qmaths.inUploaded byAnonymous ILvFoTZ
- Rtiwari Rd Book 12Uploaded bymghgol
- CS _GATE-2018_Paper_4-FEB-2018 Morning Session (1)Uploaded byAkshay Mahajan
- Chapter 1098Uploaded byJeevan Jyoti Baliarsingh
- Journal FEAUploaded bycepeng
- Matlab Getting StartedUploaded bydarebusi1
- Matrices and Row OperationsUploaded byHazel Clemente Carreon
- The network of global corporate controlUploaded byMarkoff Chaney
- Center of Mass SpecificationsUploaded byJose Mari Cipriano
- All LecturesUploaded byMuhammad Ajmal
- 2012.Ranc Pelajaran TahunanUploaded byWan Zaitul Zulkifli
- lecture3.pdfUploaded byAlemayehu Asmare
- nagarro question papers 96Uploaded byakhiles!
- UCOE-UCSC Math 23B_Calculus 4 (Spring 17) - Section 16.1.pdfUploaded byMorvaridYi
- Stat 222 r TutorialUploaded byMoonKang
- project proposal- van spronsenUploaded byapi-239505795
- LAB3Uploaded bytarek mahmoud
- Lab1Uploaded byAtif Imtiaz
- AQL Sampling TableUploaded bysherviny

- BBA 101Uploaded bySanJeev Dani PeDrosa
- 4_Bending_of_plates.docUploaded bysudarshan
- WST01_01_que_20170120Uploaded byMohammed MaGdy
- Gate SyllabusUploaded byfreakylogger
- XAct ParisUploaded bymadhavanrajagopal
- Intro to MatlabUploaded byHrishikesh Joshi
- 8 Fourier Series and TransformsUploaded bymaurizio.desio4992
- Elements of the Theory of the Newtonian Potential FunctionUploaded bysypha163
- 0606_s05_qp_2Uploaded bySwordfighterX
- DRMAC VESELIC - 2008-Siam-new Fast and Accurate Jacobi Svd AlgorithmUploaded byJulio Cesar Barraza Bernaola
- [Richard Bellman] Introduction to Matrix Analysis,(BookFi)Uploaded byPoil de Carotte
- Mean Preserving SpreadUploaded bymokus2000
- An Introd. to P-Adic Numbers & P-Adic Analysis - March 28th 2011 (Baker)Uploaded byfelipe_cruz_6
- Lecture 3 Polynomial Interpolation for UploadUploaded byRianna Cruz
- Algebra and Linear Equations _(14!8!2014)Uploaded bySudhir Kumar
- Week 1.pdfUploaded byAiman Alammari
- Module 8 - Differentiation 2 (self study)Uploaded byapi-3827096
- Splines AlternateUploaded byharris
- Srinivasa RamanujanUploaded bysunidarsana
- Tb Intro MatlabUploaded byTimothy Martens
- Unit B Activities and ReviewsUploaded byRyanFaller
- McCabe's_synthesis_hyd_packages.pdfUploaded byATQ
- IFEM.Ch30Uploaded byRafael Buss
- Existence, Uniqueness and Stability Solution of Differential Equations with Boundary ConditionsUploaded byIOSRjournal
- Sets, Relations, Functions and Number SystemUploaded byAnuj Tripathi
- Computer Vision - El robadonUploaded byMarco Souza
- Introduction to Molecular Dynamics Simulations. Prabal MaitiUploaded byCornersouth
- PermutationsUploaded byheyyaa
- Lectures Chapter 2CUploaded byShivneet Kumar
- NumericalUploaded byRachit Madan