Professional Documents
Culture Documents
ECE Home Undergraduate Home My Home Numerical Analysis Table of Contents 0 Introduction 1 Error Analysis 2 Numeric Representation 3 Iteration 4 Linear Algebra 4.1 PLU Decomposition 4.2 PLU Decomposition on Tridiagonal Matrices 4.3 Positive-definite Matrices 4.4 Cholesky Decomposition 4.5 QR Decomposition 4.6 Eigenvalues and Eigenvectors 4.7 Norms 4.8 Ill-conditioned Matrices 4.9 The Jacobi Method 4.10 The Gauss-Seidel Method 4.11 Overrelaxation Techniques 5 Interpolation 6 Least Squares 7 Taylor Series 8 Bracketing 9 The Five Techniques 10 Root Finding 11 Optimization 12 Differentiation 13 Integration 14 Initial-value Problems 15 Boundary-value Problems
Appendices
Introduction
Positive-definite matrices have some nice properties which we will exploit in finding more convenient decompositions which require less memory.
Background
Useful background for this topic includes: Matrix multiplication and inner (dot) products
References
Bradie, Section 3.7, Special Matrices, p.211. Weisstein, http://mathworld.wolfram.com/PositiveDefiniteMatrix.html.
Theory
A symmetric n n matrix M is said to be positive definite if for all nonzero vectors v, the product vTMv > 0. Looking more closely at this product, we see that it is the dot product of vT and Mv. You will recall that the dot product of two vectors is equal tovTMv = ||v||2 ||Mv||2 cos(),where is the angle between v and Mv, and and thus, for the dot product to be positive, it means that the image ofv must have an angle less than 90o, i.e., || < /2. It is unreasonable to test every possible vector v to determine if a matrix is positive definite, however, there are some cases which appear quite often in engineering where it is quite simple to state that a matrix is positive definite. These are covered on the howtos. The following are some interesting theorems related to positive definite matrices:
Theorem 4.2.1
A matrix is invertible if and only if all of the eigenvalues are non-zero. Proof: Please refer to your linear algebra text.
Theorem 4.2.2
A positive definite matrix M is invertible. Proof: if it was not, then there must be a non-zero vectorx such that Mx = 0. ThereforexTMx = 0 which contradicts our assumption aboutM being positive definite.
Theorem 4.2.3
All the eigenvalues with corresponding real eigenvectors of a positive definite matrix M are positive.
Proof: if x is an eigenvector of M then Mx = xand therefore xTMx = ||x||2. For this to be positive, it follows that > 0.
Diagonal Dominance
A matrix M is row diagonally dominant if
for i = 1, ..., n, and diagonally dominant if it is both row and column diagonally dominant. Thus, we may restate our first definition as: A symmetric row diagonally-dominant matrix with positive diagonal entries is positive definite.
Error Analysis
[an error occurred while processing this directive]
Examples Example 1
Show that if A is invertible, then ATA is positive definite. If A is invertible, then Av 0 for any vector v 0. Therefore vT(ATA)v = (vTAT)(Av) which is the vectorAv dotted with itself, that is, the square of the norm (or length) of the vector. As Av 0, the norm must be positive, and thereforevT(ATA)v > 0.
Questions Question 1
Create a random positive-definite n n matrix in which is also diagonally dominant.
rand(n, n) + (n - 1)*eye( n ) )
Question 3
Which of the following matrices are positive definite? a. b. c.
Applications to Engineering
The conductance matrix of a RLC circuit is positive definite. Additionally, we will see that the matrix defined when performing least-squares fitting is also positive definite. As an alternate example, the Hurwitz criteria for the stability of a differential equation requires that the constructed matrix be positive definite. This is seen in signals and linear systems.
Matlab
A test for positive definiteness requires that the matrix is symmetric and that all the eigenvalues are positive.
M = [...];
% assume M is square
isposdef = true; for i=1:length(M) if ( det( M(1:i, 1:i) ) <= 0 ) isposdef = false; break; end end isposdef % 0 if false, 1 if true
Maple
The following commands in Maple tests if a matrix is positive definite:
M := <<-6, 12, 2.4, 0>|<-1, 2, 10.4, 1>|<3.25, 1, -1.8, 14.8>|<10.25, 0, 2, 1.2>>; LinearAlgebra[IsDefinite]( M );
For more help on either of these routines or on the LinearAlgebra package, enter:
?LinearAlgebra ?LinearAlgebra,IsDefinite
Douglas Wilhelm Harder Department of Electrical and Computer Engineering University of Waterloo 200 University Avenue West Waterloo, Ontario, CanadaN2L 3G1 Phone: +1 519 888 4567 extension 37023 Fax: +1 519 746 3077
Contact the Author Department of Electrical and Computer Engineering Faculty of Engineering