Mathematical Programming (MP)

Concept
• Definition
Linear Programming • MP includes a range of powerful computer-
based optimization methods
• Approach to Optimization
– MP methods use special features of the structure of a
problem to get solutions efficiently
– Different MP methods use different structures or
features
– Understanding which features apply to a particular
problem and thus which method can apply is
important

Mathematical Programming (MP) Linear Programming (LP)

Concept Overview
• There are two large categories of methods • LP is a special form of mathematical programming
– Methods Valid for Convex Feasible Regions
• Linear Programming, etc • LP features
• These use a local search routine to reach global optimum – Equations must be linear
• “Keep going up/down until reach top/bottom” – Uses simple solution procedures
• Very efficient -- but not always valid – Linear algebra
– Methods Valid for Non-Convex Feasible Regions
– Very powerful
• Dynamic programming, etc
• These "enumerate" solutions to discover optimum – Can handle extremely large problems (100,000 variables, 1000's
of constraints)
• They “trim,” which means, eliminate, possible solutions because
these can be shown to be “dominated” – Useful design information through sensitivity analysis
• Computations limit applicability to special situations, but good for – Answers to "what if" questions
options analysis
 Introduction to Linear Programming
• A Linear Programming model seeks to
maximize or minimize a linear function,
subject to a set of linear constraints.
• The linear model consists of the following
components:
– A set of decision variables.
– An objective function.
– A set of constraints.
5
LP Assumptions
There are three main assumptions of LP
1. Linearity
2. Additivity
3. Non-Negativity
Standard Form of LP
• LP involves 3 parts
1. Objective function (maximize or minimize)
Max c1x1 + c2x2 + ... + cnxn
Xi known as decision variables
2. Constraints
Subject to:
a11x1 + a12x2 + ... < = > b1
a21x1 + a22x2 + ... < = > b2
a31x1 + a32x2 + ... < = > b3
3. Non-Negativity
xi > 0 for all i
1. Linearity
• Linearity of Objective Function and Constraints
–Essential Condition is:
f(kX) = k f(X)
• Implies
–Constant returns to scale (only first order terms)
–No "fixed charges" (no constants)
 2. Additivity
• f(X1, X2, ... , Xn) = f(X1) + f(X2) + ... + f(Xn)
• No interactive effects among Xi terms
• Assumes that individual segments of the problem
operate as well independently as together
Consequences of Assumptions
• Convexity of feasible region (if it exists!)
• Convex feasible region, with linear objective function, implies:
– Optimum will be on an edge of the feasible region
– Since edges are also linear, Optimum is at a corner point (can
be several in special cases)

3. Non-Negativity • Note that corner points

– Constitute small, finite set
• Xi > 0 – Defined by solution of linear equations
• No fundamental difficulties except in particular • Bottom Line: Assumptions imply the existence of an efficient solution
situations strategy

Another form in matrix and vector notations is as follows:

LP Summary
S.t.
• Linear programming is applicable to optimization
problem with linear objective function and linear
constraints set: Or

S.t. i=1,2,….,m Where

j=1,2,….,n
, , ,
 The Industries Production Problem
• Galaxy manufactures two toy doll
models:
– S.
– Z.

• Resources are limited to

– 1000 pounds of special plastic.
– 40 hours of production time per week.
14

The Industries Production Problem

The Industries Production Problem • The current production plan calls for:
• Marketing requirement – Producing as much as possible of the more profitable
product, product S ($8 profit per dozen).
– Total production cannot exceed 700 dozens.
– Use resources left over to produce product Z ($5 profit
– Number of dozens of S cannot exceed per dozen), while remaining within the marketing
guidelines.
number of dozens of Z by more than 350.
• Technological input • The current production plan consists of:
– S requires 2 pounds of plastic and S = 450 dozen 8(450) + 5(100)
– 3 minutes of labor per dozen. Z = 100 dozen
– Z requires 1 pound of plastic and Profit = $4100 per week
– 4 minutes of labor per dozen.
15 16
 A linear programming model
Management is seeking
can provide an insight and an
a production schedule
intelligent solution to this problem.
that will increase the
company’s profit.

17 18

The Galaxy Linear Programming Model The Galaxy Linear Programming Model

• Decisions variables: Max 8X1 + 5X2 (Weekly profit)

subject to
– X1 = Weekly production level of S (in dozens)
2X1 + 1X2 ≤ 1000 (Plastic)
– X2 = Weekly production level of Z (in dozens).
3X1 + 4X2 ≤ 2400 (Production Time)
• Objective Function:
X1 + X2 ≤ 700 (Total production)
– Weekly profit, to be maximized
X1 - X2 ≤ 350 (Mix)
19
Xj> = 0, j = 1,2 (Nonnegativity) 20
The Graphical Analysis of Linear
Programming Using a graphical presentation

The set of all points that satisfy we can represent all the constraints,
all the constraints of the model
is called the objective function, and the three

a types of feasible points.

FEASIBLE REGION

21 22

Graphical Analysis – the Feasible Graphical Analysis – the Feasible

Region Region
X2
X2
1000 The Plastic constraint
2X1+X2 ≤ 1000
700 Total production constraint:
X1+X2 ≤ 700 (redundant)
The non-negativity constraints 500

Infeasible
Production Feasible
X1 Time
3X1+4X2 ≤ 2400 X1
500 700

23 24
 Graphical Analysis – the Feasible Region The search for an optimal solution
8X1 + 5X2
X2
X2 Start at some arbitrary profit, say profit = $2,000...
1000 The Plastic constraint
2X1+X2 ≤ 1000 1000 Then increase the profit, if possible...
700 Total production constraint: ...and continue until it becomes infeasible
X1+X2 ≤ 700 (redundant)
500 700
Infeasible Profit
Production mix 500
=$4360
constraint:
Production Feasible X1-X2 ≤ 350
Time
3X1+4X2≤ 2400
X1
500 700
Interior points. Boundary points. Extreme points. X1
• There are three types of feasible 25 500
26

points

Summary of the optimal solution Extreme points and optimal solutions

S = 320 dozen – If a linear programming problem has an
Z = 360 dozen optimal solution, an extreme point is
optimal.
Profit = $4360
– This solution utilizes all the plastic and all the
production hours.

– Total production is only 680 (not 700).

– S production exceeds Z production by only 40

dozens. 27 28
 Example (1): (Graphical solution)
Solve the following linear programming problem:

Max Z= 3x1+5x2
S.t. 2x1-3x2 ≤ 2
x1+ x2 ≤ 7
x1+3x2 ≤ 9
x1 , x2 ≥ 0

From the figure, the optimum solution is x1 = 3.67, and

x2 = 1.78 and Z = 19.89

Example (2): (Graphical solution)

Max Z= 3x1-5x2
S.t. 3x1- 2x2 ≤ 2
x1+2x2 ≤ 6
x1- x2 ≤ 3
x1 , x2 ≥ 0

The solution of this problem is x1 = 0.67, x2 = 0, and Z = 2

 Some Technical Aspect of LP Some Technical Aspect of LP
Alternate Optima Unbounded Solution
• Consider the following • Consider the following
Max 3x + 2y Max 2x + 3y
St. 6x + 4y<24 St. x + y>3
10x + 3y<30 x - 2y>4

• This problem has alternate • There is no limit to have

optima because the large objective function
objective function has the
same slope of the first • No real-world problem can
constrain be unbounded, so check
your formulation

Sensitivity Analysis Why Sensitivity Analysis

• Sensitivity (or post optimality) analysis is • Math problem is an approximation
an attempt to study the manner in which – optimum is an approximation

the values of the decision variables in a – we need to check

mathematical model change as the • Constraints often artificial

parameters of the model change
• Sensitivity analysis is a method for dealing
with uncertainty
– Designer should question them
– Should we have different specifications?
• Situations always probabilistic
– Prices change
– Need to assess risk
 Changes in the Objective-Function
Sensitivity Parameters
Coefficients (Cj)
• Optimize Consider
Maximize f = 15x1 + 5x2
ST.
ST 2x1 + 3x2 ≤ 54
4x1 + 2x2 ≤ 40
• Changes in the objective-function coefficients • The solution to this problem is x1 = I0 x2=0 and f*
(Cj) = 150.
• Change in in the technological coefficients (aij) • Now suppose that the profitability of variable x2
• Changes in the right-hand sides bi increases by $2.5

Changes in the technological

coefficients (aij)
Consider
Maximize f = 15x1 + 5x2
ST 2x1 + 3x2 ≤ 54
4x1 + 2x2 ≤ 40
• The solution to this problem is x1 = I0 x2=0 and f*
= 150.
• Now suppose that the a11 increases by 10%
Changes in the Objective-Function
Coefficients (Cj)
Consider
Maximize f = 15x1 + 5x2
ST 2x1 + 3x2 ≤ 54
4x1 + 2x2 ≤ 40
• The solution to this problem is x1 = I0 x2=0 and f*
= 150.
• Now suppose that the profitability of variable x2
increases by $2.5

Shadow Price Definition

• Recall from Constrained Optimization:
– Shadow price = change objective function/
change in constraint
– at the optimum
– Complementary Slackness:

Either (Slack variable) or (shadow price) = 0

 Shadow Price Illustration Proactive Use of Shadow Prices
Max: Y = X1 + 4X2
s.t. X1 + X2 < 5 = b1 • Identify constraints with high S.P
X1 > 3 = b2
X2 < 3 = b3 • See if they can be changed for better
Notes:
X1 , X2 > 0
solutions
a) X1* = 3; X2* = 2; Y* = 11
b) when ∆b1 = +1 • Respond to new opportunities
∆ X2* = +1; ∆ Y*= +4; SP1 = 4
c) SP3* = 0; slack3 = 1 • Example: client changes specifications
d) when b1 > 6
slack3 = 0; SP3 ≠ 0;
SP1 = 1 < 4
• Respond to proposals for new constraints

Shadow Prices As Constraints

Sign of Shadow Prices
Change
• "Obvious Rule" (+SP with +∆b) not correct
• Correct Reasoning:
– What makes the optimum better?
• Expansion of feasible region => "Relaxation of constraints"

– What changes will increase the feasible region?

• Increase upper bound Σ j aijXj < bi
• Decrease lower bound Σ k akjXj > bk

– i.e., "Raise the roof, lower the floor."

 Shadow PriceRanges for
Range of Shadow Prices
Example
• In Linear Programming, Shadow prices are constant Max: Y = X1 + 4X2
• Until a constraint changes enough so that a new s.t. X1 + X2 < 5 = b1
constraint is binding
X1 > 3 = b2
• Results given as:
SPK = constant X2 < 3 = b3
for rL < bK < rU X1 , X2 > 0
• Outside the range: Shadow Prices
– Shadow prices decrease
SP1 = 4 3 < b1 < 6
as constraint is relaxed
– Shadow prices increase
SP2 = 4 2 < b2 < 5
as constraint is tightened SP3 = 0 2 < b3

Opportunity Cost - Definition A Possible Semantic Confusion

• Objective Function = Σ ci Xi
• Note that the Phrases “shadow price” and
• Opportunity costs associated with ci -- the coefficients
of design/decision variables “opportunity cost” have somewhat different
• At optimum, some decision variables = 0 meanings in LP and Economics literature
– These are non-optimal decision variables
• Opportunity cost is:
– Degradation of optimum per unit of non-optimal variable • The “opportunity cost” of an action in
• introduced into design economics can be interpreted as the “shadow
– A "cost" in that it is a worsening of optimum. Units may be
price” of that action on the budget…
almost anything; equal to whatever units are being optimized.
 Summary on LP Sensitivity
Analysis
• LP Optimization Programs automatically
provide important information useful for
improving/changing design
• Shadow prices -- to help redefine constraints
• Opportunity costs -- to identify critical prices

• Need to understand these quantities carefully