Lagrangean Relaxation

--- Bounding through penalty adjustment

1
 Outline
Brief introduction
How to perform Lagrangean relaxation
Subgradient techniques
Example: Set covering
Decomposition techniques and Branch
and Bound

2
 Introduction
Lagrangian Relaxation is a technique which has
been known for many years:
Lagrange relaxation is invented by (surprise!)
Lagrange in 1797 !
This technique has been very useful in
conjunction with Branch and Bound methods.
Since 1970 this has been the bounding
technique of choice ...... until the beginning of
the 90ies (branch-and-price)

3
Linear Program

4
How can we calculate lower bounds ?
We can use heuristics to generate upper
bounds, but getting (good) lower bounds is
often much harder !
The classical approach is to create a
relaxation.

5
6
7
The classical branch-and-bound algorithm use the
LP relaxation. It has the nice feature of being
general, i.e. applicable to all MIP models.

8
 Linear Program

Min:
s.t.

9
This is called the Lagrangean Lower Bound
Program (LLBP).
10
Note:
For any 0, the program LLBP provides a lower
bound to the original problem.
To get the tightest lower bound, we should solve

min cx (b Ax)

max s.t. Bx d
0
x {0,1}

(This problem is called Lagrangean Dual program.)
Ideally, the optimal value of Lagrangear Dual
program is equal to the optimal value of the original
problem. If not, a duality gap exists. 11
 Lagrangian Relaxation
What can this be used for ?
Primary usage: Bounding ! Because it is a
relaxation, the optimal value will bound the
optimal value of the real problem !
Lagrangian heuristics, i.e. generate a good
solution based on a solution to the relaxed
problem.
Problem reduction, i.e. reduce the original
problem based on the solution to the relaxed
problem.

12
 Two Problems
Facing a problem we need to decide:
Which constraints to relax (strategic
choice)
How to find the best lagrangean
multipliers, (tactical choice)

13
 Which constraints to relax
Which constraints to relax depends on two things:
Computational effort:
Number of Lagrangian multipliers
Hardness of problem to solve
Integrality of relaxed problem: If it is integral, we
can only do as good as the straightforward LP
relaxation !
(Integrality: the solution to relaxed problem is
guaranteed to be integral.)

14
 Multiplier adjustment
Two different types are given:

Subgradient optimisation
Multiplier adjustment

Of these, subgradient optimisation is the method of

choice. This is general method which nearly
always works !
Here we will only consider this Method, although
more efficient (but much more complicated)
adjustment methods have been suggested.
15
Lagrangian Relaxation

16
 Problem reformulation

Note: Each constraint ax

j ij ij bi is associated with a multiplier i.
17
 The subgradient

(subgradient of multiplier i)

18
Subgradient Optimization

19
Example: Set covering

20
 Relaxed Set coverint

How can we solve this problem to optimality ???

21
 Optimization Algorithm
The answer is so simple that we are
reluctant calling it an optimization
algorithm: Choose all xs with negative
coefficients !

What does this tell us about the strength of

the relaxation ?

22
Rewritten: Relaxed Set covering

where

23
Lower bound

24
 Comments
This is actually quite interesting: The
algorithm is very simple, but a good lower
bound is found quickly !
This relied a lot on the very simple LLBP
optimization algorithm.
Usually the LLBP requires much more
work ....
The subgradient algorithm very often
works ...
25
 So what is the use ?
This is all very nice, but how can we solve our
problem ?
We may be lucky that the lower bound is also a
feasible and optimal solution.
We may reduce the problem, performing
Lagrangian problem reduction.
We may generate heuristic solutions based on
the LLBP.
We may use LLBP in lower bound calculations
for a Branch and Bound algorithm.
26
 Homework
Starting with ZUB=6,=2 and i=0 for
i=1,2,3, perform 3 iterations of the
subgradient optimization method on the
previous example problem. Please show
the lower bound and multiplier values of
each iteration.

27