8 views

Uploaded by Shashank Singla

- TSP.pdf
- Presentation 1
- EXAM2
- The Use of Heuristics in Decision Tree Learning Optimization
- 10.1.1.26
- A Hybrid Bacterial Foraging Algorithm For Solving Job Shop Scheduling Problems
- Particle Swarm Optimization
- Square Grid Points Coveraged By
- “TRAVELING SALESMEN PROBLEM”.
- Metaheuristics - The Metaphor Exposed
- Metaheuristic
- Branch and Bound Algorithms
- Presentation
- Fan Yang Final Project Report Fixed Figure References
- Algorithms for the Design of a Multi-stage Adaptive Kanban System
- Mullers Method ProofED
- Three Modified Efficient Iterative Methods for Non-Linear Equations
- ch7.pdf
- Lecture I.pdf
- Intro to FEM in 1D 2D

You are on page 1of 43

and Heuristics

1

Quotes of the day

and we are born to it.

-- Thomas Harris

-- Donald Knuth

2

Heuristics

A heuristic is a technique designed for solving a

problem more quickly when classic methods are too

slow (from Wikipedia).

Todays lecture:

Heuristics illustrated on the traveling salesman problem.

Design principles for heuristics

Chances for practice

3

Traveling Salesperson Problem (TSP)

Find the shortest

distance tour

passing through

each node of the

network exactly

once.

cij = distance

from i to j.

http://www.math.uwaterloo.ca/tsp/

Courtesy of William Cook. Used with permission.

4

15,112 City Optimal Tour in Germany

(rotated)

5

Exercise: Try to find the best tour.

6

On solving hard problems

How did you select your tour?

it relied on visualization

perhaps you took a holistic view

solving hard problems?

7

The TSP is a hard problem

NP-hard (also NP-complete)

completeness before?

1. Yes.

2. No.

8

The TSP is a hard problem

Cannot bound the running time as less than nk for

any fixed integer k (say k = 15).

would be a polynomial time algorithm for every

NP-complete problem.

problem?

9

100 n15 vs. n!

Suppose that we could carry out 1 sextillion steps per

second (1021).

n = number of cities. (n-1)! tours.

compare time for 100 n15 steps vs. n! steps.

Remark: 100 n15 steps is NOT practically efficient.

n = 20, 3.27 seconds 2.4 milliseconds

n = 25, 1.5 minutes 4.2 hours

n = 30 24 minutes 8,400 years

n = 35 4 hours 326 billion years

10

Two types of Heuristics

scratch (starting with nothing).

Often called greedy heuristics. Each step

looks good, but it doesnt look ahead.

starts with a solution, and then tries to improve

the solution, usually by making small changes in

the current solution.

11

An easy construction heuristic:

Nearest unvisited neighbor

12

Animations from Stephen Mertens

13

Can we do better in construction?

Class exercise: try to develop a heuristic in

which we add one city at a time, but the next city

can be added anywhere in the tour (not just the

beginning or the end.)

Below is the beginning part of a tour

14

Courtesy of Stephan Mertens. Used with permission.

15

Which do you believe will give shorter

length tours:

16

Facility location problems.

Choose K facilities so as to minimize total

distance from customers to their closest facility.

example with three facilities

17

Exercise: try developing a good

solution where there are 2 facilities

18

Exercise: Develop a construction heuristic

for the facility location problem

Data: locations in a city.

cij = distance from i to j

19

Mental Break

20

Improvement heuristics

Improvement heuristics start with a feasible

solution and look for an improved solution that

can be found by making a very small number of

changes.

This will be made more formal

obtained from the other by deleting two edges

and adding two edges.

21

2-opt neighborhood search

A TSP tour T is called 2-optimal if there is no 2-

adjacent tour to T with lower cost than T.

lower cost than the current tour. If one is found,

then it replaces the current tour. This continues

until there is a 2-optimal tour.

22

An improvement heuristic: 2 exchanges.

Look for an improvement obtained by deleting

two edges and adding two edges.

3 10

T

2 1

7

8 3

4 2

4

5

6

1

9

10

9 8

6 7

5

23

After the two exchange

Deleting arcs (4,7) and (5, 1) flips the subpath from node 7 to node 5.

3 10

T1 T2

2 1 1

7

8 3 3

4 2 2

4 4

5 7

6 8

1

9 10

10 9

9 8 6

6 7 5

5

24

After the two exchange

Deleting arcs (1,3) and (2, 4) flips the subpath from 3 to 2.

3 10

T2 T3

2

7 1 1

8 3 2

4 2 3

4 4

7 7

1 8 8

10 10

9 9

9 6 6

6 5 5

5

25

After the final improving 2-exchange

Deleting arcs (7,8) and (10, 9) flips the subpath from 8 to 10.

3 10

T3 T4

2

7 1 1

8 2 2

4 3 3

4 4

7 7

1 8 10

10 8

9 9

9 6 6

6 5 5

5

26

2-exchange heuristic (also called 2-opt)

27

3-opt neighborhood

Two TSP tours are called 3-adjacent if one can be

obtained from the other by deleting three edges

and adding three edges.

adjacent tour to T with lower cost than T.

lower cost than the current tour. If one is found,

then it replaces the current tour. This continues

until there is a 3-optimal tour.

28

On Improvement Heuristics

Improvement heuristics are based on

searching a neighborhood . Let N(T) be

the neighborhood of tour T.

that can be obtained from T deleting two

arcs and inserting two arcs.

Improvement heuristic:

start with tour T

if there is a tour T N(T) with c(T) < c(T), then

replace T by T and repeat

otherwise, quit with a locally optimal solution.

29

How good are improvement heuristics?

Implementers

had to be very

clever to

achieve these

running times.

error bound

2-opt

3-opt

Facility location problems.

Class exercise. Suppose we want to solve a

facility location problem with 3 facilities. Design a

neighborhood search heuristic.

31

Using Randomization

An important idea in algorithm development:

randomization

way of using the same approach multiple times

and getting different answers. (Then choose the

best).

have an approach that is more likely to converge

to a good solution

32

On the use of randomization

Remark: 2-exchanges will behave differently

depending on the starting solution.

Start with a random tour

use the 2-exchange neighborhood until obtaining

a local optimum.

33

One difficulty: random tours are terrible.

34

2-opt heuristic starting from a random

solution.

35

Another use of randomization

Replace the nearest neighbor tour with the

following: at each iteration, visit either the

closest neighbor or the second or third closest

neighbors. Choose each with 1/3 probability.

good and may be a better starting point than a

totally random tour.

36

Other approaches to heuristics

The metaphor based approach to the design of

heuristics

simulated annealing

genetic algorithms

neural networks

ant swarming

That is, look for something that seems to work

well in nature, and then try to simplify it so that it

is practical and helps solve optimization

problems.

37

Simulated Annealing

A randomization heuristic based on neighborhood

search that permits moves that make a solution worse.

It is based on an analogy with physical annealing.

and have it reach a low

Photo removed due to energy state, one

copyright restrictions.

should cool it slowly.

A glass annealing oven.

www.carbolite.com

European Journal of Operational Research 46 (1990) 271-281. (PDF)

38

Simulated Annealing: a variation on local search.

and a starting temperature T = T0

( c( x)-c( y))/T

If c(y) > c(x), replace x by y with probability e

(e.g., T := .99T)

Yes

Is T still No

large enough Quit.

39

A typical acceptance rate

as a function of temp

Min Fang. All rights reserved. This content is excluded from our Creative Commonslicense. For more

information, see http://ocw.mit.edu/help/faq-fair-use/.

Source: Fang, M. Ch. 3 in "Layout Optimization for Point-to-Multi-Point Wireless Optical Networks Via

Simulated Annealing & Genetic Algorithm." Senior Project Report, University of Bridgeport, 2000.

40

A typical graph of Temp vs. cost.

Min Fang. All rights reserved. This content is excluded from our Creative Commonslicense. For more

information, see http://ocw.mit.edu/help/faq-fair-use/.

Source: Fang, M. Ch. 3 in "Layout Optimization for Point-to-Multi-Point Wireless Optical Networks Via

Simulated Annealing & Genetic Algorithm." Senior Project Report, University of Bridgeport, 2000.

41

Summary

Two types of heuristics

Use of randomization

Simulated annealing

hard problems.

42

MIT OpenCourseWare

http://ocw.mit.edu

Spring 2013

For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.

- TSP.pdfUploaded byLibyaFlower
- Presentation 1Uploaded bysachu03
- EXAM2Uploaded byGillian Amba
- The Use of Heuristics in Decision Tree Learning OptimizationUploaded byIJCERT PUBLICATIONS
- 10.1.1.26Uploaded byRobin Cook
- A Hybrid Bacterial Foraging Algorithm For Solving Job Shop Scheduling ProblemsUploaded byijplajournal
- Particle Swarm OptimizationUploaded bySHELL
- Square Grid Points Coveraged ByUploaded byAnonymous Gl4IRRjzN
- “TRAVELING SALESMEN PROBLEM”.Uploaded bysuryakantshrotriya
- Metaheuristics - The Metaphor ExposedUploaded byArdeshir Bahreininejad
- MetaheuristicUploaded byHafisIzran
- Branch and Bound AlgorithmsUploaded byMEdrifter
- PresentationUploaded byAnurag Yadav
- Fan Yang Final Project Report Fixed Figure ReferencesUploaded byNguyen Tuan Linh
- Algorithms for the Design of a Multi-stage Adaptive Kanban SystemUploaded byKharisma Elan
- Mullers Method ProofEDUploaded byHAMM3T
- Three Modified Efficient Iterative Methods for Non-Linear EquationsUploaded byivy_publisher
- ch7.pdfUploaded byRaul Soles
- Lecture I.pdfUploaded byjegosss
- Intro to FEM in 1D 2DUploaded byB S Praveen Bsp
- 1 DIAMOL 2 D4 Sequence AlignmentUploaded byRa
- Task 3C FinalUploaded byShravan
- Detail SylUploaded byFelixberto Baguyo Jr.
- lec06_PUploaded byrashom0099088
- Particle Swam CAUploaded byNguyen Anh
- Networkk Reliability LarbiUploaded byKenshin Himura
- 6 Numerical Methods for Constrained Optimization.pdfUploaded byAugusto De La Cruz Camayo
- Section 6 IntegrationUploaded bypravesh
- VRPUploaded bycsrajmohan2924
- Gaussian Elimination PPT LectureUploaded byKim Lee

- MIT15_053S13_lec8.pdfUploaded byShashank Singla
- MIT15_053S13_lec9.pdfUploaded byShashank Singla
- MIT15_053S13_lec19.pdfUploaded byShashank Singla
- MIT15_053S13_lec12.pdfUploaded byShashank Singla
- MIT15_053S13_lec13.pdfUploaded byShashank Singla
- MIT15_053S13_lec15.pdfUploaded byShashank Singla
- MIT15_053S13_lec16.pdfUploaded byShashank Singla
- MIT15_053S13_lec14.pdfUploaded byShashank Singla
- MIT15_053S13_lec18.pdfUploaded byShashank Singla
- MIT15_053S13_lec10.pdfUploaded byShashank Singla
- MIT15_053S13_lec7.pdfUploaded byShashank Singla
- MIT15_053S13_lec4.pdfUploaded byShashank Singla
- MIT15_053S13_lec5.pdfUploaded byShashank Singla
- MIT15_053S13_lec1.pdfUploaded byShashank Singla
- MIT15_053S13_lec6.pdfUploaded byShashank Singla
- MIT15_053S13_lec2Uploaded byShashank Singla
- Lscm IndiaUploaded byShashank Singla
- MIT15_053S13_lec11.pdfUploaded byShashank Singla
- MIT15_053S13_lec3.pdfUploaded byShashank Singla
- New Business StrategyUploaded byShashank Singla
- New Business StrategyUploaded byShashank Singla
- Corporate ValuationUploaded byShashank Singla

- 2008_06_20_NASA_HDBK_6003C_APPROVEDUploaded byxc0de
- Kinetics LabUploaded byAcee Echevarria
- a m projectUploaded byteohjoshua
- Maxima and Minima of Functions of Two VariablesUploaded byRaj Arun
- matrics modulUploaded byFadzilah Yaya
- mbqt 4-5Uploaded byapi-288833495
- PC235W13 Assignment1 SolutionsUploaded bykwok
- CH 5 Simple MechanismsUploaded byPatel Nikhil
- MasteringPhysics Ch 21 HW #2Uploaded byGiancarlo Santos
- The BigChaos Solution to the Netﬂix Grand PrizeUploaded byapi-25884893
- Learning Simio Lab SeriesUploaded bybasemfallatah
- I love rituUploaded byᕠᙏᓰᖶ ᐺᕦᖆᘻᗩ
- Homework of Correlation and RegressionUploaded bydip171
- Modeling Cracked Behavior of Shear Walls in ETABSUploaded byMoy Erga
- M.Sc.(CS)Uploaded bymuralikrish14u
- CSC108_TT2_2010FUploaded byexamkiller
- ch10Uploaded byC.P. Wong
- Aubt for HtatUploaded byseso20081
- chapt06(magnetic compass)Uploaded byapi-20017870
- GATE Signals & Systems BookUploaded byFaniAli
- Speedcrunch Manual (3)Uploaded byAghil Vinayak
- Forex Report - Predicting Price MovementUploaded bymohora
- akaroa-omnetUploaded byArie Wirawan
- NEET AIPMT Gravitation Sample Paper 02Uploaded byramchandra
- GEESDIV2008_compinclUploaded byeltopo2
- Instantaneous Spectral BandUploaded bydayutttt
- Blood Lead Levels in School Children in Chennai, IndiaUploaded bydrtpk
- mttc basic skillsUploaded byapi-297289928
- M433 Center of Mass Location Throughout Fuze Arming CycleUploaded byNORDB
- Clustering With RUploaded byAdrian Iosif