1

1. INTRODUCTION
1.1. DISCRETE OPTIMIZATION PROBLEMS
Discrete optimization or combinatorial optimization means searching for an optimal solution in a
finite or countably infinite set of potential solutions. Optimality is defined with respect to some
criterion function, which is to be minimized or maximized.
Examples of
minimization: cost, distance, length of a traversal, weight, processing time, material, energy
consumption, number of objects
maximization: profit, value, output, return, yield, utility, efficiency, capacity, number of objects.
The solutions may be combinatorial structures like arrangements, sequences, combinations, choices
of objects, sequences, subsets, subgraphs, chains, routes in a network, assignments, schedules of
jobs, packing schemes, etc.
Example 1.1.
A sales rep sells 12 types of vacuum cleaners. During one day he has to visit 9 customers by car to
demonstrate his products.
Problem 1:
What is the shortest route? ("Traveling Salesman problem")
Problem 2:
All the 12 models do not fit in the car. What models to take? ("Knapsack Problem")
Given data:
- distances between locations
- trunk size of the car and sizes of models
- expected profit / utility for demonstrating each model
A graph for problem 1:
Home: node 0
Customers: nodes 1,...,9
The optimization problem can be formulated in a generic form
(OPT) minimize f(x)
subject to x 0 S
where f: D 6 is the objective function (or criterion function), D is the domain of f and S f D is
the set of feasible solutions x, defined according to some limitations, requirements or logical con-
straints.
Definition 1.1.
Solution x* 0 S is an optimum solution of (OPT) if f(x*) # f(x) for all x 0 S.
2
In the maximization case the inequality is reversed. The optimum solution is also called minimum
solution or maximum solution accordingly. The problem may have more than one optimum
solution.
In discrete optimization the set D is a discrete, countable set. In contrast to continuous sets and
continuous functions, the concept of a local minimum cannot be defined unless a suitable metric
(or distance between the solutions) is defined. Such a metric is highly problem dependent and it
can be used to define a neighborhood structure in the solution space.
1.2. GRAPH THEORETIC CONCEPTS
A graph is a pair G =(V,E), where V is a set of vertices or nodes and E is a set of edges, each
joining two vertices. We denote the vertices by numbers 1,...,n (sometimes v ,...,v ) and the edge
1 n
between vertex i and vertex j by [i,j] or i-j. The edges have no direction, thus [i,j] =[j,i].
Example 1.2. In the following graph
V ={1,2,3,4,5,6}
E ={[1,2], [1,3], [2,3], [2,5], [2,6], [3,4], [4,5], [5,6]}.
Vertices i0V and j0V are adjacent (or neighbors) if [i,j]0E. Then we say that edge e=[i,j] is
incident to i and j. The degree of a vertex is the number of edges incident to it.
Different data structures can be used for representing a graph: a list of vertices and edges, an
adjacency matrix, an incidence matrix, an adjacency list (neighbor list).
If the number of vertices |V| =n, the adjacency matrix is a symmetric nn matrix A =(a),
ij
a = 1 if [i,j]0E
ij
9 0 otherwise
If there is more than one edge between two vertices, the graph is a multigraph.
In a weighted graph each edge [i,j] has a weight w associated with it. The weights can represent
ij
e.g. distances, costs, or capacities.
A directed graph or digraph is a graph D =(V,A) where each edge has a direction. These directed
edges are called arcs. Arc from node i to node j is denoted by (i,j) or i6j.
A mixed graph has both directed and undirected edges.
A walk in a graph G=(V,E) is a sequence of adjacent nodes, denoted by w =[v ,v ,...,v ] or
1 2 k
w =v -v -...-v such that [v,v ]0E for i=1,...,k-1.
1 2 k i i+1
The walk can also be thought of as a sequence of edges [v ,v ], [v ,v ],..., [v ,v ].
1 2 2 3 k-1 k
The walk is closed if v =v (k>1). A closed walk is called a circuit.
1 k
3
A walk with no repeated nodes is a path.
A walk w =[v ,v ,...,v ] with no repeated nodes except v =v is a cycle.
1 2 k 1 k
Directed walk, path and cycle are defined accordingly, using notation w =(v ,v ,...,v ) or
1 2 k
w =v 6v 6...6v .
1 2 k
A graph is complete if there is an edge between every pair of vertices i.e. if all vertices are adjacent
to each other.
A graph is connected if there is a path between every pair of vertices.
A graph is planar if it can be drawn in the plane without crossing edges.
A tree is a connected graph with no cycles.
A Hamiltonian path is a path that includes every vertex exactly once. A Hamiltonian cycle is a
cycle that includes every vertex exactly once.
An Eulerian walk / circuit is a walk / closed walk, in which each edge appears exactly once.
Graph modeling can be used in optimization problems encountered e.g. in telecommunications,
data transmission, transportation and logistics, services and distribution networks, wiring, neural
networks.
1.3. EXAMPLES
(1) TRAVELING SALESMAN PROBLEM (TSP)
Find a minimum weight Hamiltonian cycle in a given (complete) weighted graph.
Application: A salesman has to visit n cities and come back home. Distances between cities are
given. Find the shortest route.
Remark: The application is not analogical to the original problem if the road network is not a
complete graph because the shortest route is not necessarily a Hamiltonian cycle.
(2) POSTMAN PROBLEM (or CHINESE POSTMAN PROBLEM)
In a weighted graph, find a minimum weight closed walk that includes every edge at least once.
Application: A postman has to deliver mail to all streets in a certain neighborhood. Find the
shortest route.
Remark: The graph may be undirected, directed or a mixed graph.

(3) KNAPSACK PROBLEM
Given n objects with values v and weights w, i=1,...,n and maximum capacity M of the knapsack.
i i
Choose a subset of objects I f {1,...,n} such that the total weight does not exceed the capacity M
and the total value is maximized.
(4) BIN PACKING PROBLEM
Given n objects with weights w, i=1,...,n and maximum capacity M of a bin. The objects are to be
i
packed to bins so that the number of bins is minimized.
(5) SET COVERING PROBLEM
Given a finite set S and subsets S f S with associated costs c, i=1,...,n. Choose a collection of
i i
subsets {S | i 0 I} with index set I f {1,...,n} such that and the total cost is mini-
i
mized.
4
(6) MAXIMUM CLIQUE PROBLEM
In a given graph G =(V,E), determine the maximum clique i.e. the greatest set of vertices that are
pairwise adjacent i.e. the vertices are completely connected.
(7) SCHEDULING PROBLEM
Given n jobs are to be completed using m machines. Every job consist of a sequence of operations
on different machines. The total schedule of operations is to be determined such that the total
makespan of all operations is minimized.
Remarks:
In addition to the machine sequence and time requirements, also a due date for each job may be
given. Alternative objectives: minimize the sum of completion times of all jobs, sum of delays,
maximum delay, number of late jobs etc.
(8) ASSIGNMENT PROBLEM
Given n tasks and n machines capable of performing any of the tasks. Costs c of performing the
ij
task j with machine i are given. Find the minimum cost assignment of tasks to machines.
Alternatively:
Given compatibilities c between objects of two equal sets. Find the maximum total compatibility
ij
assignment.
(9) MINIMUM SPANNING TREE PROBLEM
In an undirected weighted graph, find the minimum cost spanning tree i.e. a connected acyclic
subgraph that includes all the vertices.
(10) SHORTEST PATH PROBLEM
In a given weighted graph, find the shortest path from node i to node j.
(11) MINIMUM WEIGHT PERFECT MATCHING
In a given weighted graph G=(V,E), find the set of edges FfE with minimum weight such that no
two edges have the same end point and |F| =|V|/2.
(12) MINIMUM WEIGHT EDGE COVER
In a given weighted graph G=(V,E), find the set of edges FfE with minimum weight such that each
vertex i0V is an end point of at least one edge of F.
(13) MINIMAL VERTEX COVER
In a given graph G=(V,E), find the smallest number of vertices SdV, such that every edge is
incident to at least one vertex of S, i.e.
{[i,j] * i0S, j0V} =E.
(14) VERTEX COLORING, CHROMATIC NUMBER
Color the vertices of a graph with a minimum number of colors such that adjacent vertices have
different colors.
The most interesting problem in discrete optimization is whether there is an algorithm that solves
the given problem in a polynomial time with respect to the size of the problem instance. If such an
efficient algorithm exists, the problem is considered easy or "tractable". The problems (8)-(12) in
the preceding list are known to be tractable, whereas for the problems (1)-(7), (13)-(14) no
polynomial time algorithm is known (except for special cases). Efficiency of algorithms and the
computational or algorithmic complexity of problems is considered in Chapter 2.
5
1.4. FORMULATION OF INTEGER PROGRAMMING PROBLEMS
All of the previously mentioned problems can be modeled as an integer linear programming model
of the form
(ILP) minimize
subject to
for i=1,...,m
x $ 0 for j=1,...,n
j
x integer for j0 I.
j
Constraints may also be of the form ...#b or ...=b. Variables may be stated as binary, if they take
i i
values 1or 0.
Without the integer requirement this problem would be a linear programming problem (LP) and
solvable with well known methods. However, it is often advisable to take into consideration the
special combinatorial structure of the problem instead of using a general algorithm based on this
ILP-formulation. For some problems the ILP-model may contain an immense number of variables
and/or constraints.
Example 1.3. (Assignment Problem)
Given n tasks and n machines capable of performing any of the tasks. Costs c of performing the
ij
task j with machine i are given. Find the minimum cost assignment of tasks to machines.
Define the decision variables
1, if machine i performs task j
x =
ij
9 0, otherwise
A binary integer linear programming model
min
subject to
for every i=1,...,n
for every j=1,...,n
x =0 or 1 for every i,j=1,...,n
ij
Example 1.4. (Traveling Salesman Problem)
Find the shortest Hamiltonian cycle in a given complete graph G=(V,E) with n vertices and.distan-
ces c =distance from vertex i to vertex j, i,j =1,...,n
ij
6
Decision variables:
1 if the cycle goes along arc i j
x =
ij
9 0 otherwise
In the assignment model part of the restrictions of the TPS model are assured but the solution may
consist of separate cycles. Extra constraints are needed to prevent separate, unconnected cycles.
Optimization model (Tucker formulation):
min
subject to
for every i=1,...,n (every vertex has one successor)
for every j=1,...,n (every vertex has one predecessor)
u - u +nx # n-1 for every i,j =2,...,n, i=/j (prevents cycles that don't include
i j ij
vertex 1)
x =0 or 1 i,j=1,...,n
ij
u 0 i=2,...,n
i
Other model formulations for the TSP problem are given in Chapter 5.
1.5. ON SOLUTION METHODS
The main categories of solution methods:
Exact methods are guaranteed to find the optimal solution in finite computational time
Heuristic methods (or heuristics) have no guarantee for optimality but they are based on reason-
able, justifiable principles or rules of thumb, leading to good or near-optimal solutions. These
methods are often easy to implement and efficient.
Heuristic methods are preferred over exact methods, when
the problem is so large or high dimensional that the exact methods might take too long
the exact methods - if they exixt - require the problem to be coded in a format that is too compli-
cated, extensive or time consuming
no codes are available for exact algorithms, programming would be too tedious, time consuming,
expensive
a feasible and functional solution is needed quickly
problem data is uncertain, inaccurate or fuzzy exact solution isn't of much worth.
Evaluation criteria for heuristics:
1) Quality of solution:
How close to the optimum is the value of the objective function?
Is the quality dependent on the size or structure of the problem, accuracy of problem parameters
etc?
7
2) Cost of solution:
How long does it take to get the solution?
How much work, time and other resources are required? (Modeling, programming, computing
time, space requirements etc)
An approximation algorithm is an algorithm that produces a solution such that its objective
function value is within specified distance from the optimum.
An instance of a problem is an individual case of a certain problem, where the input parameters are
fixed.
Let x*(I) denote an optimal solution of an instance I of a certain problem.
Definition 1.2. An approximation algorithm A produces a g-approximation, if the following
condition holds for the solution x(I) :
for all instances I.
For minimization with a positive objective function, the condition can be written
f(x(I)) # [1 +g(I)] f(x*(I)).
If e.g. g(I)=1, the approximation algorithm always produces a solution with an objective value at
most twice the optimum. Function g is usually dependent on the size of the instance I. An ideal
approximation algorithm would have a constant bound g(I)=g that can be made arbitrarily small.
We shall assume in the following that the set of solutions D (or feasible solutions S) is finite. This
is the case in practically all combinatorial problems (e.g. all the problems mentioned above).
"Brute force" method: Enumerate all the solutions, check feasibility, calculate objective values and
keep the best. This is an exact method but suitable only for very small instances.
Examples:
A traveling salesman has to visit n cities exactly once (starting from and returning to the first city).
The number of different cycles that pass through every city exactly once is (n-1)!/2 (in undirected
case). This function has an enormous rate of growth: for n=20 the number of solutions is . 6@10 .
16
So the brute force approach is out of question.
Many combinatorial problems (e.g. the Knapsack problem) can be modeled as a decision problem
with binary variables. If the solution is specified with e.g. 200 independent yes/no choices, the total
number of solutions is 2 . 10 .
200 60
The most common general strategy for discrete optimization problems is the Branch and bound
method. This exact method is suitable for a general integer programming problems and can be
tailored for different problem classes. The method is based on a "divide and conquer" principle: the
problem is partitioned or divided into subproblems and bounds for the objective value are calcula-
ted for each subproblem. The branching produces a tree structure. The complete enumeration is
avoided by pruning or discarding the nonoptimal branches by means of comparing the bounds.
8
GENERAL PRINCIPLES FOR THE DESIGN OF HEURISTICS:
1. CONSTRUCTION
The solution is built up piece by piece, by adding elements to a partial arrangement.
Example: a TSP tour is constructed from a chosen initial vertex by adding vertices or arcs one by
one according to a specified principle.
2. IMPROVEMENT
An initial feasible solution (e.g. a TSP tour) is modified in order to achieve an improvement in the
objective value (a shorter tour). The trial solutions are usually generated from specified neighbor-
hood of the current solution. This is called a local search (or neighborhood search).
3. PARTITIONING
The problem is partitioned into smaller parts. These parts are solved separately and the final
solution is assembled from the parts.
Example: In TSP, the vertex set of a graph is partitioned e.g. by a clustering method. The TSP tours
of the subgraphs are constructed and finally joined to form a total solution.
4. DECOMPOSITION
The problem is solved in subsequential phases or stages. The solution of the current phase is based
on the previous one. This strategy is used in dynamic programming.
5. RANDOM SEARCH
Trial solutions can be generated randomly or the current solution can be modified randomly, using
appropriate probability distributions.
6. PARALLEL SEARCH
Several solutions are considered simultaneously: improving, modifying and comparing them with
each other. This is the general principle of genetic algorithms.
7. MIXED STRATEGIES
Combinations of different principles:
- Construct an initial solution and try to improve it.
- If local improvement is impossible, make a random modification.
- Generate a random initial solution and improve it. Repeat. The probability of finding the optimum
solution increases with the number of repetitions.