An Ant Colony Optimization Approach to the Border Penetration Model

Philipp A. Djang Ph.D. ARL/SLAD/IEPD December 12, 2002

Overview
Introduction to Ant Colony Algorithms
An Application to Border Penetration

Introduction
First proposed by M. Dorigo, 1992 Heuristic optimization method inspired

by biological systems Multi-agent approach for solving difficult combinatorial optimization problems
Traveling

Salesman, vehicle routing, sequential ordering, graph coloring, routing in communications networks

Has become new and fruitful research

area

Ant Colony Algorithms

Algorithm was inspired by observation

of real ant colonies. Ants are essentially blind, deaf and dumb. Ants are social creatures behavior directed to survival of colony Q: how can ants find the short path to food sources? Ants deposit pheromones on ground that form a trail. The trail attracts other ants.

Ant Colony Algorithms

Pheromone mediated following

behavior induces the emergence of shortest paths. Probability of choosing a branch of a path at a certain time depends on the total amount of pheromone on the branch. The choice is proportional to the number of ants that have used the branches.

Ant Colony Algorithms

Let um and lm be the number of ants that

have used the upper and lower branches. The probability Pu(m) with which the (m+1)th ant chooses the upper branch is:

( k ) u m P ( m) (u m k ) (l m k )
h u h

Ant Colony Algorithms

Ant behavior is a kind of stochastic

distributed optimization behavior. Although one ant is capable of building a solution, it is the behavior of an ensemble of ants that exhibits the shortest path behavior. The behavior is induced by indirect communication (pheromone paths) and is termed stigmergy.

Ant Colony Algorithms

Each ant collects information about

local environment; acts concurrently and independently No direct communication: stigmergy paradigm governs information exchange Incremental constructive approach to building solutions High quality solutions emerge via global cooperation.

Ant Colony Algorithms

Ants do not know the global structure of

the problem - discover the network Limited ability to sense local environment - can only see adjacent nodes of immediate neighborhood. Each ant chooses an action based on variable probability
random

choice pheromone mediated

Algorithm: Overview
Initialize ants: pick start and goal nodes

For each ant do:

Move:

Select a node in local neighborhood

Randomly choose a node pheromone mediated (greedy)

Communicate:

deposit scent trail If goal node is found, increase pheromone weights of path Check Time-to-Live: ant dies if time is exceeded.

Ant Colony Algorithms: Summary

Ant Colony Algorithms mimic Real Ants
Colony

of cooperating individuals Simulated Pheromone Trail and Stigmergy Shortest path searching with local moves Stochastic and myopic state transition policy
Artificial ants:
Discrete

state transitions Pheromones based on solution quality Pheromone laying is problem dependent

Border Penetration Problem

An application of an Ant Colony Optimization Algorithm

Border Penetration Problem

A group of terrorists must travel from

destination city in Canada to a target city in the US. A fixed number of locations and routes that link the locations are given. For each route, a difficulty rating and risk rating are assigned. The problem is to find the shortest path given the risk and difficulty of each route.

Border Penetration Problem

However, this is a stochastic problem.

From any location, the agent selects the next location and a fair die is rolled. If the value of the die is less than the difficulty rating, the agent waits one turn, otherwise the agent proceeds. If the value of the die is less than the risk rating, the agent lives, otherwise, the agent dies.

Border Penetration Problem

Formally, the problem can constructed as: Given a Graph G with Nodes (N) and a set of

edges (E). G is incompletely connected. Let i,j N, a set of Nodes And i* = start node and j* = end node Let xij =1 if the edge between node i and j are selected in the optimal route. Let Dij represent an estimate of the difficulty for edge i->j and ij be realized value Let Rij represent an estimate of the risk for edge i->j and ij be realized value

Border Penetration Problem

The problem is to find the minimum

route path subject to difficulty and risk constraints

x x d x r
j ij ij ij ij ij ij ij ij

Min z i

ij ij

ij

Border Penetration Problem

An ant colony optimization algorithm

was developed to address this stochastic problem. A software simulation system was implemented to visualize the penetration of the border. And illustrate how the system discovers different routes and eventually finds the best routes.

Border Penetration Problem

Ants have a limited time to live and are

removed when
The

risk policy forces death Time to live (simulation steps) is exceeded Dead ants are replaced at the next time step
Ants select nodes stochastically: With probability p, a random node is selected With probability 1-p, the pheromone trail will influence the selection of a node.

Border Penetration Problem

Better routes are stored in artificial

pheromone memory and used to bias the selection of future nodes The selection probability for an edge changes as better routes are discovered. The simulation allows the user to select any start and end node; ants discover the graph and construct paths from the start to the end node. The simulation records descriptive statistical behaviors of the ant colony.

A Screen Shot

Future Work
Develop a blue agent system to protect

against red agent penetration. Blue agent adjusts the risk factor of the edges subject to resource constraints. Blue agent decisions will be based on reflexive control concepts Interaction between red and blue agents may give yield co-evolutionary strategy development

Conclusion
The ant colony algorithm is can be

generalized to other problems. For example, if the ants can be considered and mobile (disposable) unattended ground sensors, the algorithm could be used to guide them to find interesting objects. The algorithm could also be used to assist with intelligent movement of tactical vehicles.