Optimization using Calculus

Kuhn-Tucker
Conditions

1 D Nagesh Kumar, IISc Optimization Methods: M2L5

 Introduction

Optimization with multiple decision variables and equality

constraint : Lagrange Multipliers.
Optimization with multiple decision variables and inequality
constraint : Kuhn-Tucker (KT) conditions
KT condition: Both necessary and sufficient if the objective
function is concave and each constraint is linear or each constraint
function is concave, i.e. the problems belongs to a class called the
convex programming problems.

2 D Nagesh Kumar, IISc Optimization Methods: M2L5

 Kuhn Tucker Conditions: Optimization
Model

Consider the following optimization problem

Minimize f(X)
subject to
gj(X) 0 for j=1,2,,p
where the decision variable vector

X=[x1,x2,,xn]

3 D Nagesh Kumar, IISc Optimization Methods: M2L5

 Kuhn Tucker Conditions

Kuhn-Tucker conditions for X* = [x1* x2* . . . xn*] to be a local minimum are

f m
g
+ j =0 i = 1, 2,..., n
xi j =1 xi
j g j = 0 j = 1, 2,..., m
gj 0 j = 1, 2,..., m
j 0 j = 1, 2,..., m

4 D Nagesh Kumar, IISc Optimization Methods: M2L5

 Kuhn Tucker Conditions contd.

In case of minimization problems, if the constraints are

of the form gj(X) 0, then j have to be non-positive
On the other hand, if the problem is one of maximization
with the constraints in the form gj(X) 0, then j have to
be nonnegative.

5 D Nagesh Kumar, IISc Optimization Methods: M2L5

 Example (1)

Minimize f = x1
2
+ 2 x2
2
+ 3 x3
2

subject to

g1 = x1 x2 2 x3 12
g 2 = x1 + 2 x2 3 x3 8

6 D Nagesh Kumar, IISc Optimization Methods: M2L5

 Example (1) contd.
Kuhn Tucker Conditions
f g g 2 x1 + 1 + 2 = 0 (2)
+ 1 1 + 2 2 = 0
xi xi xi 4 x2 1 + 22 = 0 (3)
6 x3 21 32 = 0 (4)

j g j = 0 1 ( x1 x2 2 x3 12) = 0 (5)
2 ( x1 + 2 x2 3 x3 8) = 0 (6)

x1 x2 2 x3 12 0 (7)
gj 0 x1 + 2 x2 3 x3 8 0 (8)

1 0 (9)
j 0 2 0 (10)

7 D Nagesh Kumar, IISc Optimization Methods: M2L5

 Example (1) contd.

From (5) either 1 = 0 or x1 x2 2 x3 12 =, 0

Case 1
From (2), (3) and (4) we have x1 = x2 =2 / 2 and x3 = 2 / 2
Using these in (6) we get 2 + 82 = 0, 2 = 0 or 8
2

From (10), 2 0 , therefore, 2 =0,

Therefore, X* = [ 0, 0, 0 ]
This solution set satisfies all of (6) to (9)

8 D Nagesh Kumar, IISc Optimization Methods: M2L5

 Example (1) contd.

Case 2: x1 x2 2 x3 12 = 0
Using (2), (3) and (4), we have 1 2 1 22 21 + 32 12 = 0
2 4 3
or 171 + 122 = 144

But conditions (9) and (10) give us 1 0 and 2 0 simultaneously,

which cannot be possible with 171 + 122 = 144

Hence the solution set for this optimization problem is X* =[000]

9 D Nagesh Kumar, IISc Optimization Methods: M2L5

 Example (2)

Minimize f = x12 + x22 + 60 x1

subject to
g1 = x1 80 0
g 2 = x1 + x2 120 0

10 D Nagesh Kumar, IISc Optimization Methods: M2L5

 Example (2) contd.
Kuhn Tucker Conditions
f g g 2 x1 + 60 + 1 + 2 = 0 (11)
+ 1 1 + 2 2 = 0
xi xi xi 2 x2 + 2 = 0 (12)
1 ( x1 80) = 0 (13)
j g j = 0 2 ( x1 + x2 120) = 0 (14)
x1 80 0 (15)
gj 0 x1 + x2 + 120 0 (16)
1 0 (17)
j 0
2 0 (18)
11 D Nagesh Kumar, IISc Optimization Methods: M2L5
 Example (2) contd.

From (13) either 1 = 0 or ( x1 80) = 0 ,

Case 1
From (11) and (12) we have x1 = 2 30 and x2 =
2
2 2
Using these in (14) we get 2 ( 2 150 ) = 0
2 = 0 or 150
Considering 2 = 0, X* = [ 30, 0]. But this solution set violates (15)
and (16)
For 2 = 150 , X* = [ 45, 75]. But this solution set violates (15)

12 D Nagesh Kumar, IISc Optimization Methods: M2L5

 Example (2) contd.

Case 2: ( x1 80) = 0

Using x1 = 80 in (11) and (12), we have

2 = 2 x2
1 = 2 x2 220 (19)

Substitute (19) in (14), we have 2 x2 ( x2 40 ) = 0

For this to be true, either x2 = 0 or x2 40 = 0

13 D Nagesh Kumar, IISc Optimization Methods: M2L5

 Example (2) contd.

For x2 = 0 , 1 = 220

This solution set violates (15) and (16)

For x2 40 = 0, 1 = 140 and 2 = 80

This solution set is satisfying all equations from (15) to (19) and hence

the desired

Thus, the solution set for this optimization problem is X* = [ 80 40 ].

14 D Nagesh Kumar, IISc Optimization Methods: M2L5

 Thank you

15 D Nagesh Kumar, IISc Optimization Methods: M2L5