Professional Documents
Culture Documents
Instead of using , , and , we use q0, z0 and w0, respectively. Thus we have w0 = q0 + 0 z0 d Tz 0, z0 0, z0w0 = 0 w = q + dz0 + M z 0, z 0, z Tw = 0.
We assume that the constant column q 0 and the covering vector d > 0. Then the smallest positive z 0 such that q + dz0 0 for all z0 z 0 is given by z 0 = max{qi/di}.
i
Recall the theorem stating that the ALCP always has a solution. The algorithm we are about to study proves this constructively under the assumption that the system of equations w0 w is nondegenerate. = q0 q + 0 d T d M z0 z
). If q 0, then stop: z = 0 solves Step 0. Initialization. Input ( q, M (q, M ). Otherwise, let z 0 = maxi{qi/di}. Let wr denote the (unique, by the nondegeneracy assumption) component of w that equals zero when z0 = z 0. Pivot wr , z0 . Choose the driving variable to be the complement of wr , namely zr . Step 1. Determination of the blocking variable. Use the minimum ratio test to nd the basic variable that blocks the increase of the driving variable. Stop if w0 is the blocking variable. (Interpret this outcome, if possible.)
Step 2. Pivoting. The driving variable is blocked. If z0 is the blocking variable, then pivot z0, driving variable and stop. A solution to (q, M ) is at hand. If some other variable blocks the driving variable, then pivot blocking variable, driving variable . Return to Step 1 using the complement of the most recent blocking variable as the new driving variable.
After the rst pivot in Lemkes algorithm, all basic variables (and only basic variables) are eligible to block the driving variable. But why does blocking always occur?
Lemma.
in which a 0, < 0, and B is nonsingular. Then the pivotal obtained by using B as the pivot block has at least transform of M one negative entry in its last column.
Proof. The indicated pivotal transform is
Theorem.
In Lemkes Algorithm, when w0 and z0 are basic, the column of the driving variable contains at least one negative entry.
Proof. Consider the w0 row and the set of rows corresponding to basic
z -variables (including the articial variable, z0). Let be the index set of z -variables (including z0) that are currently basic. Let be the index set of the w-variables that are currently nonbasic. These index sets have the same cardinality, but dier by just one element, r: will contain the index 0, and will contain the index of the most recent blocking variable (which is now nonbasic). The of the lemma corresponds to M . The remaining submatrix B in M corresponds to M r of the current driving variable. column of M This is illustrated in the next panel.
aT B c
1 z0
aT B 1 aTB 1c B 1 B 1 c z zr z
T T w0 q0 0 d drT d
M r M w q d M M r M w q d M
T , aT = 0 d
= drT
B = d M
r. c=M
To initialize the algorithm, it is enough to choose the column vector d so as to make the n 2 matrix [d, q ] have lexicographically nonnegative rows. Nevertheless, it is customary to let d = e, the vector of ones. The choice of the covering vector can aect the number of iterations of the algorithm.
Making z0 z 0 and z = 0 yields a set of feasible solutions to the ). ALCP ( q, M These points constitute what is called the primary ray. Points along the primary ray satisfy the conditions z0w0 > 0 and ziwi = 0 for all i = 0. Thus, they are almost complementary in the sense that complementary slackness holds for all but one index. In between almost complementary extreme points, we have almost complementary edges.
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
w = q + dz0 + M z 0, z0 0, z 0, z Tw = 0. We denote this system by (q, d, M ). A solution of (q, d, M ) with z0 = 0 furnishes a solution to the original LCP (q, M ). The version of Lemkes Scheme I for (q, d, M ) is a simple variant of ). the one for ( q, M
Secondary rays
), we have to Without the constraint w0 0 that is used in ( q, M allow for the possibility that the column of a driving variable may be nonnegative and hence that the variable may be unblocked. Such a situation corresponds to the case where q0 is large and w0 is the blocking variable in the earlier version of Lemkes Algorithm for the full augmented LCP. In the event that no blocking variable is found when a driving variable is increased, a secondary ray is generated. A secondary ray is also an almost complementary edge of FEA(q, d, M ), but it is necessarily unbounded.
(a) The n-vector e whose components are all 1 is often used as the default covering vector. Other choices are possible, however. The choice of the covering vector can drastically aect the outcome of the procedure. (b) After the initial pivot which makes the articial variable z0 basic, there is one (and only one) complementary pair of nonbasic variables, initially, wr and zr are such a pair. In general, the nonbasic complementary variables make up the nonbasic pair. (c) When z0 z 0, all the basic variables are nonnegative and are eligible to block the driving variable. In general, after a pivot occurs, the new driving variable is the complement of the variable that just became nonbasic.
(d) One can implement the algorithm in a revised simplex method fashion. Only two columns are needed to execute the minimum ratio test in Step 1. These are the updated constant column and the updated column of the current driving variable. Thus, if the current basis is known, these updated columns can be determined by solving two systems of linear equations. (e) There are certain classes of matrices for which the nonexistence of a blocking variable implies the infeasibility of the given LCP (q, M ).
1 z0 z1 w1 3 w2 6 w3 1
z2
z3
1 0 1 2 1 2 0 2 1 1 1 0
1 w1 z1 z2 z3 z0 w2 w3 3 9 2 1 0 1 2 1 1 1 2 1 4 2 2
1 w1 w3 z2 z3 z0 3 w2 13 z1 2 1 0 3 2 1 1 1 2 5 8 2 2
Example Continued
1 w1 w3 z0 w2 z3
z2
z1
1 0 1 1 1 5 1 2 3 4 1 1 1 1 1 2 2 2 1 w2 w3 z2 z1
z0 w1 z3
1 0 5 1 3 1 2
1 1 2 3 1 1 2 2
1 4
3 2
1 w2 w3 z2 w1 z3
z0 z1 1 1 1
1 0 1 1 2 1 1 3 1 3 1 0 2 2
When applied to a nondegenerate instance of (q, d, M ), Lemkes Algorithm (Streamlined) terminates after nitely many steps with either a secondary ray or else a complementary feasible solution of (q, d, M ) and hence with a solution of (q, M ).
Proof. This is just like the proof of niteness in the unstreamlined case.
Termination on a secondary ray corresponds to the situation where the latter algorithm terminates with w0 as the blocking variable.
Consequences of termination on a secondary ray Theorem. If Lemkes Algorithm applied to (q, d, M ) terminates with a secondary ray, then M reverses the sign of some nonzero nonnegative vector; that is )i 0 i = 1, . . . , n z i (M z
for some nonzero nonnegative vector z .
Suppose the algorithm reaches an almost-complementary extreme point (basic ) at which we have z 0 > 0 and w i z i = 0 for all i > 0. feasible solution) (w, z 0 , z If the driving variable is unblocked, we have termination on a secondary ray. ) of Hence there is a nonzero solution (w, z 0, z w = dz0 + M z, w 0 , z0 0 , z 0 .
Moreover, we have (w + w )i ( z + z )i = 0 for all i = 1, . . . , n and all 0. z 0 , z ) is zero). But z 0 > 0 implies Now z = 0, for otherwise z 0 > 0 (else (w, w > 0 and hence that the secondary ray is the primary ray, which is impossible. For i = 1, . . . , n and all 0, we have i + w i z i + w i z i + 2 w i z i = 0 w i z which implies w i z i = w i z i = w i z i = w i z i = 0 Thus for each i = 1, . . . n, i = z i di z 0 + z i (M z )i . 0=z i w Since z i di z 0 0, we have the asserted result: there exists a nonnegative nonzero vector z such that )i 0 for all i = 1, . . . , n. z i (M z
Lemkes Algorithm will solve any nondegenerate LCP (q, M ) such that M E.
Theorem.
In particular, Lemkes Algorithm will solve any nondegenerate LCP (q, M ) such that M P or M SCP (strictly copositive).
Proof. It is impossible for a strictly semimonotone matrix to reverse
the sign of a nonzero, nonnegative vector. Recall that a matrix M Rnn is said to be semimonotone if [0 = x 0] [xk > 0, (M x)k 0 for some k ] The class of semimonotone matrices is denoted E0.
Theorem.
Let M Rnn E0. If Lemkes Algorithm applied to (q, d, M ) terminates with a secondary ray, then SOL(0, M ) = {0}.
on a ray, we know that there exists a nonzero nonnegative vector ( z0, z ) such that w = dz 0 + M z 0 and z Tw = 0. Since M E0 we cannot have z 0 > 0, for otherwise, 0 = w = d z 0 + M z where = supp( z ). Hence M z < 0. But now it is clear that z SOL(0, M ).
If M CP and Lemkes Algorithm applied to (q, d, M ) terminates with a secondary ray, then the vector y = z / z satises: (i) 0 = y SOL(0, M ), (ii) q Ty < 0.
holds. As for (ii), note that for all 0 we have + M ( z + z )) ( z + z )T(q + dz 0). 0 = ( z + z )T(q + dz Dividing through by z + z and letting we obtain y T(q + dz 0) 0. Since y Tdz 0 > 0 we obtain (ii).
Theorem.
If M is copositive and q (SOL(0, M )), then Lemkes Algorithm will compute a solution of the LCP (q, M ) if the problem is nondegenerate.
Constructive proof that CP+ Q0 Theorem. If M CP+ and Lemkes Algorithm applied to a nonde-
generate instance of the problem (q, d, M ) terminates on a secondary ray, then the LCP (q, M ) is infeasible.
Proof. If the algorithm terminates on a secondary ray, we have
0=z Tw =z Tdz 0 + z TM z . Since M is copositive, both terms on the right are nonnegative, hence z = 0. both are zero. But since M CP+, we have (M + M T) We also know that z 0 = 0, hence M z 0 and M Tz 0. The latter is equivalent to z TM 0. Moreover z Tq < 0; thus z 0, z TM 0, z Tq < 0.
Can Lemkes Algorithm nd global minima of nonconvex QPs? Example. Consider the quadratic program
minimize subject to
1 2 x1 1 2 1 2 1 2 x2 2 x1 + 2 x2
2 x1 + x2 6 x1 + 4x2 6 x1 0 , x 2 0
The problem has a nonempty compact constraint set, hence a global optimum. The KKT conditions give the LCP (q, M ) where 1 1 0 2 1 2 0 1 and M = q= 6 2 1 6 1 4
2 1 1 0 0
4 0 0
1 y1 y2 x3 x4
1 2 1 2
z0
x1
x2 0 1
y3
y4
d1 1 1 0
2 1 1 0 0 4 0 0
6 d3 2 1 6 d4 1 4
1 y1 y2 x3 x4
1 2 1 2
z0
x1
x2 0 1
y3
y4
d1 1 1 0
2 1 1 0 0 4 0 0
6 d3 2 1 6 d4 1 4
y2
x1
x2 d1 1 4 d4
y3
y4
+ d1 )
1 2
d1 1 1 0 1
+ d3 ) d3 2 1 d3 + d4 ) d4
1 y1 z0 x3 x4
1 2 (12 1 2 (12 1 2 (1
y2
x1
x2 d1 1 4 d4
y3
y4
+ d1 )
1 2
d1 1 1 0 1
+ d3 ) d3 2 1 d3 + d4 ) d4
y2 0 1 4
x1 1 0 1
z0 d1 1 4 + d4
y3 2 1 4
y4 1 4 16
1 4
1 2 1 + d3
1 y1 x2 x3 x4
1 2 1 2 11 2
y2 0 1 4
x1 1 0 1
z0 d1 1 4 + d4
y3 2 1 4
y4 1 4 16
1 4
1 2 1 + d3
The solution of the KKT conditions found is (x1 , x2 , y3 , y4 ) = (0, 1 2 , 0, 0). 1 The point (x1 , x2 ) = (0, 2 ) is a local minimum. The global minimum is (x1 , x2 ) = (3, 0). It cannot be reached by Lemkes Algorithm, no matter what covering vector is used.