You are on page 1of 76

# Chapter 13

Nonlinear Models:
Dynamic, Goal and
Nonlinear Programming

## 13.1 Introduction to Nonlinear

Programming
Most real-life situations are more
accurately depicted by nonlinear
models than by linear models.
Nonlinear models formulated in this
chapter fall into three broad categories:

## Dynamic programming models (DP).

Goal programming models (GP).
General nonlinear programming models
(NLP).

## Dynamic programming problems can be thought

of as multistage problems, in which decisions are

## Known applications of dynamic programming in

Resource allocation,

Equipment replacement,

Reliability.

state.

## A decision made at stage n takes the system to

stage n+1, and leaves it in a new state for a stagestate related cost/profit.

## The decision makers challenge is to find a set of

optimal decisions for the entire process.

## 13.5 Goal Programming

In real life decision situations, virtually all
problems have several objectives.
When objectives are conflicting, the
optimal decision is not obvious.
Goal programming is an approach that
seeks to simultaneously take into account
several objectives.

## Goals are prioritized in some sense, and their level of

aspiration is stated.

## An optimal solution is attained when all the goals are

reached as close as possible to their aspiration level, while
satisfying a set of constraints.

## Nonpreemtive goal programming - no goal is pre-determined to

dominate any other goal.
Preemtive goal programming - goals are assigned different priority
levels. Level 1 goal dominates level 2 goal, and so on.

## A nonlinear programming problem (NLP) is one in

which the objective function, F, and or one or more
constraint functions, Gi, possess nonlinear terms.

## There is no a universal algorithm that can find the

optimal solution to every NLP.

## One class of NLPs Convex Programming Problems

can be solved by algorithms that are guaranteed to
converge to the optimal solution.
7

## Properties of Convex Programming

Problems
The objective is to maximize a concave
function or to minimize a convex function.
The set of constraints form a convex set.

Function

## A smooth function (no sharp points, no

discontinuities)
One global maximum (minimum).
A line drawn between any two points on the
curve of the function will lie below (above) the
curve or on the curve.
X
A convex function

A Concave function

## An illustration of a two variable

convex function

10

Convex Sets
If a straight line that joins any two points in
the set lies within the set, then the set is
called a convex set.

Convex set

Non-convex set
11

## In a NLP model all the constraints are of

the less than or equal to form G i(X) B.

## If all the functions Gi are convex, the set

of constraints forms a convex set.

12

Types of NLP
Constrained
Unconstrained

13

## Examples for NLP Problems

Dynamic Programming
US Department of Labour
Goal Programming
Non Linear Programming
Toshi Camera
PBI

14

## THE U.S. DEPARTMENT OF LABOR

(example for a dynamic
programming problem)

15

## The U.S. Department of Labor has made up to

\$5 million available to the city of Houston for job
creation.
Requests for funding are to be prepared by four
departments.
The Department of Labor would like to allocate
funds to maximize the number of jobs created.

16

## THE U.S. DEPARTMENT OF LABOR

Data
Estimated new jobs created for proposal
Department
Department
Housinng
Housinng
Employment
Employment

Highway
Highway
Law
Law Enforcement
Enforcement

Cost
Cost \$\$ millions
millions Jobs
Jobs Created
Created
44
225
225
11
45
45
22
125
125
33
190
190
50
50 jobs
jobs per
per \$1
\$1 million
million funding
funding
22
75
75
33
155
155
44
220
220

17

SOLUTION

## Fund at most one proposal from each

department
18

SOLUTION

Notation
Dj = the amount allocated to department j, where j is:
1 - housing, 2 - Employment, 3 - Highway, 4 - Law Enforcement.
R(Dj) = the number of new jobs created by funding department j with
\$Dj million.

The model

Non-linear functions

## Max R1(D1) + R2(D2) + R3(D3) + R4(D4)

ST
D1 + D2 + D3 + D4 <= 5
D1, D2, D3 , D4 >= 0

19

## Define Fj(Xj) as the maximum number of new

jobs created by departments (stages) j, j+1,
, 4, given that there is \$Xj million in funding
available for departments j (state) through 4.

20

## Stage 4: The Law Enforcement,(LED)

department, LED).
Allocate to this department funds that maximize the
number of new jobs created
Obviously, the optimal solution for the last department
is to use all the amount available at this stage).
The optimal solution for the last stage is called The
boundary condition

21

SOLUTION

## Stage 4: The Law Enforcement table

Recall
Law
LawEnforcement
Enforcement

States
Cost
Cost
22
33
44

Jobs
Jobs
7575
155
155
220
220

Available
Available Optimal
Optimal Funding
Funding Maximum
Maximum jobs
jobs
Funding
F4(X4)
Funding For
For Stage
Stage 44
F4(X4)
00
00
00
11
00
00
22
22
75
75
33
33
155
155
44
44
220
220
55
55
220
220

22

SOLUTION

## At this stage we consider the funding of both the

Highway department and the Law Enforcement
department.
For a given amount of funds available for funding of
these two departments, the decision regarding the
funds allocated to the HD affects the funds available
for the LED).

23

SOLUTION
Stage 3: The Highway Department table
Available
Funding
for Stage 3,4
(X3)
0
1
2

Possible
Funding
for Stage 3
(D3)
0
0
1
0
1
2
0
1
2
3

Remaining
Funds for
Stage 4
(X3-D3)
0
1
0
2
1
0
3
2
1
0

## Max. New Jobs Optimal F3(X3)

When Allocating
D3 to Stage 3
Optimal D3
R3(D3)+F4(X3-D3)
0+0
F3(0) = 0; D3(0) = 0
0+0
50+0 = 50
F3(1) = 50; D3 = 1
0+75 = 75
50+0 = 50
100+0
F3(3) = 100; D3 = 2
0+155 = 155
50+75 = 125
100+0 = 100
150+0 = 150
F3(3) = 155; D3 = 0

24

SOLUTION
Stage 3: The Highway Department table
Available
Funding
for Stage 3,4
(X3)

Possible
Funding
for Stage 3
(D3)

0
1
2
3
4
0
1
2
3
4
5

Remaining
Funds for
Stage 4
(X3-D3)

4
3
2
1
0
5
4
3
2
1
0

## Max. New Jobs Optimal F3(X3)

When Allocating
D3 to Stage 3
Optimal D3
R3(D3)+F4(X3-D3)

0+220 = 220
50+155 = 205
100+75 = 175
150+ 0 = 150
200+ 0 = 220 F3(4) = 220; D3 = 0
0+220 = 220
50+220 = 270
100+155 = 255
150+75 = 225
200+0 = 200
250+0 = 250 F3(5) = 270; D3 = 1

25

SOLUTION

## At this stage we consider the funding of both the

Employment Department and the previous
departments (HD and the LED)

## For a given state (the amount of funds available

for funding of these three departments), the
decision regarding the funds allocated to the ED
affects the funds available for the HD and LED.
(the state at the stage j=3)
26

SOLUTION
Funding
Funding
Funds for
for Stage 2,3,4 for Stage 2 Stage 3,4
(X2)
(D2)
(X2-D2)
0
1
2
3

0
0
1
0
1
2
0
1
2
3
0
1
2
3
0
1
2
3

0
1
0
2
1
0
3
2
1
0
4
3
2
1
5
4
3
2

When Allocating
D2 to Stage 2
Optimal D2
R2(D2)+F3(X2-X2)
0+0
0+50 = 50
45+0 = 45
0+100 = 100
45+50 = 95
125+0 = 125
0+155 = 155
45+100 = 145
125+50 = 175
190+0 = 190
0+220 = 220
45+155 = 195
125+100 = 225
190+50 = 240
0+270 = 270
45+220 = 265
125+155 = 280
190+100 = 290

F2(0) = 0; D2 = 0
F2(1) = 50; D2 = 0
F2(2) = 125; D2 = 2

F2(3) = 190; D2 = 3

F2(4) = 240; D2 = 3

F2(5) = 290; D2 = 3

27

SOLUTION

## At this stage we consider the funding for the

Housing Department and all previous
departments.

## Note that at stage 1 we know there is exactly \$5

million to allocate
(X1 = 5).
28

SOLUTION

## Stage 1: The Housing Department submitted

only one proposal of \$4 million, thus
(D1 = 0, 4).
Do not fund

Available
Possible
Funding
Funding
for Stage 1,2,3,4 for Stage 1
(X1)
(D1)
5

0
4

Remaining
Funds for
Stage 2,3,4
(X1-D1)
5
1

## Max. New Jobs

Optimal F1(X1)
When Allocating
D3 to Stage 3
Optimal D1
R1(D1)+F2(X1-D1)
0+290 = 290
225+50 = 275

F1(5) = 290; D1 = 0

29

SOLUTION
Stage
1
2
3
4

D2 = 0
D3 = 0
D4 = 0

D2 = 0
D3 = 1
D4 = 1

State
2

D2 = 2
D3 = 2
D4 = 2

D2 = 3
D3 = 0
D4 = 3

D2 = 3
D3 = 0
D4 = 4

5
D1 = 0
D2 = 3
D3 = 1
D4 = 5

In
In summary,
summary, the
the optimal
optimal funds
funds allocation
allocation
to
to maximize
maximize the
the number
number of
of jobs
jobs created
created is:
is:
Housing
Housing
Employment
Employment
Highways
Highways
Law
Law Enforcement
Enforcement
Maximum
Maximum number
number of
of jobs
jobs created
created == 290
290

== \$0
\$0
== \$3
\$3 million
million
== \$2
\$2 million
million
== \$0
\$0
30

## Components of Dynamic Programming

Stage Variable
State Variable

Notation
j
Xj

Decision Variable

Dj

## Stage Return Values

Rj(Dj)

Optimal Value

Fj(Xj)

Boundary Condition

F(XN)

Optimal Solution
Value

F1(T)

Description
A decision point
The amount of resource
left to be allocated
A possible decision
at stage j
The stage return for
making decision Dj
The best cumulative
return for stages j and
remaining stages at state Xj.
A set of optimal values
for the last stage (N)
The best cumulative
return

Example
The four departments
Funds left to allocate
to departments
Funds that could have
been give to department j
Number of jobs created
at department j if funding Dj
Maximum number of new
jobs created for dept j,,4.
The new jobs created from
funding Dept 4 with \$X4 million.
The maximum total number
of new jobs given \$Tmillion to
allocate to all departments

31

## Bellmans principle of optimality.

From a given state at a given stage, the optimal solution for the
remainder of the process is independent of any previous

## The following recursive relationship describes the process for

the U.S. Department of Labor problem.
Define Fj(Xj) as the maximum number of new jobs created by
departments (stages) j, j+1,, 4, given that there is \$X j million
in funding available for departments j (state) through 4.

all feasible Xj

32

## The form of the recursion relation differs from

problem to problem, but the general idea is the
same:

## Do the best you can for the remaining

stages with the remaining resources
available.

33

(Goal Programming)

## If these were constraints rather than goals we

would have:
3000X1 + 800X2 + 250X3 25,000
1000X1 + 500X2 + 200X3 30,000
X1
10

## No feasible solution exists that satisfies all

the constraints.
When these constraints are simply goals they
are to be reached as close as possible.
34

Detrimental variables
Ui = the amount by which the left hand side
falls short of (under) its right had side
value.
Ei = the amount by which the left side
exceeds its right had side value.

## The goal equations

3000X1 + 800X2 + 250X3 + U1 E1 = 25,000
1000X1 + 500X2 + 200X3 + U2 E2 = 30,000
X1
+ U3 E3 = 10
35

## The objective is to minimize the penalty of

not meeting the goals, represented by the
detrimental variables
E1, U2, U3.
25,000

30,000

10
36

## The penalties are estimated to be as follows:

\$25,000 cost the company \$1.

## There is a loss of \$5 to the company for each

customer not being reached, below the goal of
30,000.

## Each television spot below 10 is worth 100 times

each dollar over budget.
37

The goal programming model

## It is assumed that no advantage is gained by

overachieving a goal.
Minimize 1E1 + 5U2 + 100U3
s.t.
3000X1 + 800X2 + 250X3 + U1 E1 = 25,000
1000X1 + 500X2 + 200X3 + U2 E2 = 30,000
X1
+ U3 E3 = 10
All variables are non-negative.
38

## Minimize (the total weighted deviation)

Subject to
Expresion of the goal Equation
Functional Constraint
Non Negativity Constraint

Minimize
ST

E1 + 5U2 +100 U3

## 3000X1 + 800 X2 + 250 X3 + U1 E1

=25000
1000X1 + 500 X2 + 200 X3
+U2 E2
= 30000
X1
+U3 E3=10

39

## Preemptive Goal Prog

7 Goals
2 in priority level 1
3 in priority level 2
2 in priority level 3
Detrimental deviation

Weights

W1, W2
W3,W4,W5
W6, W7

Level 1 Goals
Level 2 Goals
Level 3 Goals

40

## Preemptive Goal Prog

(Problem Formulation)
Priority 1 Minimize
ST

Priority 2 Minimize
ST

Priority 2 Minimize
ST

W1E1 + W2U2
Goal Equation
Functional Constraint
Non Negativity Constraint
W3E3 + W4U4 + W5E5
Goal Equation
Functional Constraint
W1E1 + W2U2 = V1 (new Constraint)
Non Negativity Constraint
W6U6 + W7E7
Goal Equation
Functional Constraint
W1E1 + W2U2 = V1
W3E3 + W4U4 + W5E5 = V2 (New Constraint)
Non Negativity Constraint

41

## The NECC is planning next month production of

its two bicycles B2 and S10.
Data

## Both models use the same seats and tires.

2000 seats are available; 2400 tires are available.
1000 gear assembly are available (used only in the
S10 model).
Production time per unit: 2 hours for B2; 3 hours for
S10.
Profit: \$40 for each B2; 10\$ for each S10.

42

## Priority 1: Fulfill a contract for

400 B2 bicycles to be delivered
next month.

## Priority 2: Produce at least

1000 total bicycles during
the month.

Priority 3:
Achieve at least
\$100,000 profit for the
month.
Use no more than 1600
labor-hours during the
month.

Priority 4:
At least 200 tires left
over at the end of the
month.
At least 100 gear
assemblies left over at
the end of the month.

43

## Management wants to determine the

production schedule that best meets its
prioritized schedule.

44

NECC - SOLUTION

Decision variables
X1 = The number of B2s to be produced next month
X2 = The number of S10s to be produced next month

## Functional / nonnegativity constraints

2X 1 X 2 2000
X 2 1000
2X 1 2X 2 2400

Seats
Gear assemblies
Tires

X 1 ,X 2 0

45

NECC - SOLUTION

Goal constraints
Priority 1 (goal 1): Production of at least 400 B2s
X1
+ U1 - E1 = 400
Priority 2 (goal 2): Production of at least 1000 total cycles
X1 +
X2
+ U2 - E2 = 1000
Priority 3 (goal 3) Profit of at least \$100,000
.04X1 + .10X2 + U3 - E3 = 100 (in \$1000)
Priority 3 (goal 4) Use a maximum of 1600 labor hours
2X1 + 3X2 + U4 - E4 = 1600
Priority 4 (goal 5) At least 200 leftover tires
2X1 + 2X2 + U5 - E5 = 2200
Priority 4 (goal 6) At least 100 leftover gear assembly
X2
+ U6 - E6 = 900

46

NECC - SOLUTION

## Priority level objectives

Priority 1: Underachieving a production of 400 B2s:
Minimize U1
Priority 2: Underachieving a total production of 1000:
Minimize U2

47

NECC - SOLUTION

## Priority 3: Underachieving a \$100,000 profit

more than 1600 labor-hours
Minimize 30U3 + E4

Using

## Each \$1,000 short of the \$100,000 goal is considered

30 times as important as utilizing an extra labor-hour.

48

NECC - SOLUTION

## Priority level objectives

Priority 4: Using more than 2200 tires
Using more than 900 gear assemblies
Minimize E5 + 2E6
Each leftover gear assembly is deemed
twice as important as leftover tire.

49

## Solve the linear goal programming for priority 1 objective, under

the set of regular constraints and goal constraint as shown

Minimize U1

below
ST
X1 = 400, thus
U1 = 0, and
priority 1 goal is
fully achieved.

2X 1 X 2 2000

Seats

X 2 1000

Gear

2X 1 2X 2 2400

Tires

X1 + U1 - E1 = 400
50

## NECC - The solution procedure

Solve the linear goal programming for priority 2 level objective,
under the set of original constraints plus the constraint X1 400
(maintain the level of achievement of the priority 1 goal).
Minimize U2
ST
Every point that
2X 1 X 2 2000
Seats
satisfies X1 + X2
X 2 1000
Gear
1000 yields U2 = 0,
and therefore,
2X 1 2X 2 2400
Tires
priority 2 goal is fully
achieved.
X1
400

X1 + X2 + U2 - E2 = 1000
51

## Solve the linear goal programming for priority 3

level objective, under the set of original constraints
plus the constraint X1 400 (maintain the level of
achievement of the priority 1 goal), plus the
constraint X1 + X2 1000 (maintain the level of
achievement of the priority 2 goal).
Every point in the range X1 = 400 and 600 X2 800 is
optimal for this model; 30U3 + E4 = 1720 is the level of
achievement for the priority 3 goal.

52

## Solve the linear goal programming for priority 4

level objective, and notice that after the previous
step the feasible region is reduced to a segment of
a straight line between the points (400,600) and
(400,800).
X1 =400; 600 X2 700 and E5 + 2E6 = 0

53

## In summary NECC should produce

400 B2 model
Between 600 and 700 S10 model

54

## Global View on NLP problems

Unconstraint Models
Single Variable
Necessary & Sufficient Conditions (df/dx = 0 , D2f/dx2 <> 0))

Multi Variable
Partial Derivative

Constraint Models
Single Variable
Optimality Criteria

Multi Variable
Kuhn Tucker Condition

55

Programming

## One-variable unconstrained problems are

demonstrated by the Toshi Camera
problem.

## The inverse relationships between

demand for an item and its value (price)
are utilized in this problem.
56

## TOSHI CAMERA (example for

unconstrained NLP)

## Toshi camera of Japan has just

developed a new product, the Zoomcam.
It is believed that demand
for the initial product will
be linearly related to the
price.

Price
P (\$)
100
150
200
250
300
350

Estimated
Demand (X)
350,000
300,000
250,000
200,000
150,000
100,000

57

TOSHI CAMERA

## Unit production cost is estimated to be \$50.

What is the production quantity that
maximizes the total profit from the initial
production run?
SOLUTION
Total profit = Revenue - Production cost
F(X) = PX - 50X

58

TOSHI CAMERA
From the Price / Demand table it can be verified
that P = 450 - .001X
The Profit function becomes
F(X) = (450 - .001X)X = 400X - .001X2

400,000

59

TOSHI CAMERA

## To obtain an optimal solution (maximum profit),

two conditions must be satisfied:

## A necessary condition dF/dX = 0

A sufficient condition d2F/dX2 < 0.

## The necessary condition is satisfied at:

dF/dX = 400 - 2(.001)X = 0; X = 200,000.

d2F/dX2 = -.002.

## Produce 200,000 cameras.

The profit is F(200,000) = \$40,000,000.

60

## Optimal solutions for concave/convex

functions with one variable
If a function is known to be concave or
convex at all points, the following condition is
both a necessary and sufficient condition for
optimality:
The point X* gives the maximum value for a
concave function, or the minimum value for a
convex function, F(X), if at X*
dF/dX = 0
61

## Optimal solutions for concave/convex

functions with more than one variable
Determining whether or not a multivariate
function is concave or convex requires analysis
of the second derivatives of the function.
A point X* is optimal for a concave (convex)
function if all its partial derivatives are equal to
zero at X*.
For example, in the three variable case:

FF11

0;
XX11 0;

FF22

0;
XX22 0;

FF33

0;
XX33 0;

62

## 13.8 Constrained Nonlinear

Programming Problems one variable

## The feasible region for a one variable

problem is a segment on a straight line
(X a or X b).

## When the objective function is nonlinear

the optimal solution must not be at an
extreme point.
63

## Toshi Camera needs to determine the

optimal production level from among the
following three alternatives:
150,000 X 300,000
50,000 X 175,000
150,000 X 350,000

64

## TOSHI CAMERA solution

The objective function does not change: Maximize F(X) = 400X - .001X2
X 150,000
X 300,000

150

X 50,000
X 175,000

400

50

175

X*=200,000

400

X* = 175,000
X* = 250,000

X 250,000
X 350,000

400

65

## Constrained Nonlinear Programming

Problems m variables
MaximizeF( X1,X 2 , ..., Xn )
ST
G1(X1, X 2 , ..., Xn ) B1
G2(X1, X 2 , ..., Xn ) B 2
.
Let us define Y1, Y2, ,Ym
.
as the instantaneous improvement
.
in the value of F for one unit
Gm(X1, X 2 , ..., Xn ) Bm
increase in B1, B2, Bm
respectively.
66

## This is a set of necessary conditions for optimality of

most nonlinear problems.
If the problem is convex, the K-T conditions are also
sufficient for a point X* to be optimal.

## S1, S2, ,Sm

are defined as
the slack variables
in each constraint.

1. X * isfeasible.
2. Y1, Y2 , ..., Ym 0
3.Y1S1 0, Y2 S 2 0,..., Ym Sm 0
4.F

Y G2

... Ym Gm
Y1 G1

X1
X1
X
Xm

.
.
.
F

Y 2 G2
... Y Gm

Y1 G1

m
Xn
X1
X 2
Xm

67

PBI INDUSTRIES

## PBI wants to determine an optimal production

schedule for its two CD players during the month
of April.
Data

## There is additional intermix cost of \$0.01(the

number of portable CDs)(the number of deluxe CDs).
68

PBI INDUSTRIES

## Forecasts indicate that unit selling price for

each CD player is related to the number of
units sold as follows:

## Portable CD player unit price = 150 - .01X1

Deluxe CD player unit price = 350 - .02X2

69

PBI INDUSTRIES

Resource usage

## Each portable CD player uses 1 unit of a particular

electrical component, and .1 labor hour.
Each deluxe CD player uses 2 units of the
electrical component, and .3 labor hour.

Resource availabilty

## 10,000 units of the electrical component units;

1,500 labor hours.

70

## PBI INDUSTRIES SOLUTION

Decision variables
X1 - the number of portable CD players to produce
X2 - the number of deluxe CD players to produce

The model
Max (150-.01X1)X1+ (350-.02X2)X2 - 50X1- 90X2-.01X1X2 =
-.01X12 .02 X 2 2 .01X1X2 100 X1 260 X2
ST
X1 2 X 2 10,000
.1X1 +.3X2 1,500
- X1

0
X2
0

Resource constraints
Production cannot be negative
71

## For a point X1, X2 to be optimal, the K-T conditions require that:

Y1S1 = 0; Y2S2 = 0; Y3S3 = 0; Y4S4 = 0, and

Y 4 G 4
Y G2
Y G3

Y1 G1

2
3
X1

X
1
1
1
1

or
.02 X1 .01X 2 100 Y1(1) Y2 (.1) Y3 ( 1) Y4 (0)
F

Y G4
Y G2
Y G3

Y1 G1

2
3
4
X 2
X 2
X 2

X
2
2

or
.01X1 .04 X2 260 Y1(2) Y2 (.3) Y3 (0) Y4 ( 1)

72

Assume X1>0

## The assumption implies S3>0 and S4>0.

Thus, Y3 = 0 and Y4 = 0.

## Add the assumption that S1 = 0 and S2 = 0.

X1== 00 and
From the first two constraints we have X1
X2 = 5000.

73

and S2 > 0.

## As before, from the first assumption S3 > 0 and S4 > 0.

Thus, Y3 = 0 and Y4 = 0.
From the second assumption Y2 = 0.
Substituting the values of all the Ys in the partial derivative
equations we get the following two equations:
-.02X1 - .01X2 + 100 = Y1
-.01X1 - .04X2 + 260 = 2Y1
Also, since S1 = 0 (by the second assumption)
X1 + 2X2 = 10,000
Solving the set of three equations in three unknowns we get:

74

## PBI INDUSTRIES SOLUTION

X1 = 1000, X2 = 4500, Y1 = 35

## This solution is a feasible point (check the constraints).

X1 and X2 are positive.
1X1 + 2X2 <= 10,000
.1X1 + .3X2 <= 1,500

## [1000 + 2(4500) = 10,000]

[.1(1000) + .3(4500) = 1450]

## This problem represents a convex program since

It can be shown that the objective function F is concave.
All the constraints are linear, thus, form a convex set.

75

## PBI INDUSTRIES Excel SOLUTION

PBI INDUSTRIES

April Production

Portable
1000

Deluxe
4500

Electrical Components
Labor Hours

Portable
1
0.1

Deluxe
2
0.3

Total Profit
810000
Used
10000
1450

Available
10000
1500

76